2024-09-13T15:33:15.755 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T15:33:15.778 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T15:33:15.878 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903753 branch: wip-rishabh-testing-20240910.095512-reef description: fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/quincy 1-upgrade}} email: null first_in_suite: false job_id: '7903753' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'install ceph/quincy latest ' name: rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240910.095512-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: bluestore warn on legacy statfs: false bluestore warn on no per pool omap: false mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on osd down out interval zero: false mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - scrub mismatch - ScrubResult - wrongly marked - \(POOL_APP_NOT_ENABLED\) - \(SLOW_OPS\) - overall HEALTH_ - \(MON_MSGR2_NOT_ENABLED\) - slow request sha1: 50bbe7f395997dcd871813949c49addbbf630361 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 50bbe7f395997dcd871813949c49addbbf630361 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240910.095512-reef sha1: 50bbe7f395997dcd871813949c49addbbf630361 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 seed: 1152 sha1: 50bbe7f395997dcd871813949c49addbbf630361 sleep_before_teardown: 0 subset: 984/1024 suite: fs suite_branch: wip-rishabh-testing-20240910.095512-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_50bbe7f395997dcd871813949c49addbbf630361/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 50bbe7f395997dcd871813949c49addbbf630361 targets: smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGYfV3u+lofsJ2Y5eqDVRbxn2FHE5sUjgnZA8eB2AzM+gitm8GrJxaSI77j03KVieidlNDdKexVafxYub7tmiug= tasks: - install: branch: quincy exclude_packages: - librados3 - ceph-mgr-dashboard - ceph-mgr-diskprediction-local - ceph-mgr-rook - ceph-mgr-cephadm - cephadm - ceph-volume extra_packages: - librados2 - print: '**** done installing quincy' - ceph: conf: global: mon warn on pool no app: false ms bind msgr2: false log-ignorelist: - overall HEALTH_ - \(FS_ - \(MDS_ - \(OSD_ - \(MON_DOWN\) - \(CACHE_POOL_ - \(POOL_ - \(MGR_DOWN\) - \(PG_ - \(SMALLER_PGP_NUM\) - Monitor daemon marked osd - Behind on trimming - Manager daemon - exec: osd.0: - ceph osd set-require-min-compat-client quincy - print: '**** done ceph' - print: '*** upgrading, no cephfs present' - exec: mon.a: - ceph fs dump - install.upgrade: mon.a: null - print: '**** done install.upgrade' - ceph.restart: daemons: - mon.* - mgr.* mon-health-to-clog: false wait-for-healthy: false - ceph.healthy: null - ceph.restart: daemons: - osd.* wait-for-healthy: false wait-for-osds-up: true - exec: mon.a: - ceph versions - ceph osd dump -f json-pretty - ceph fs dump - ceph osd require-osd-release reef - for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done - ceph.healthy: null - print: '**** done ceph.restart' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_14:36:41 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T15:33:15.879 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_50bbe7f395997dcd871813949c49addbbf630361/qa; will attempt to use it 2024-09-13T15:33:15.879 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_50bbe7f395997dcd871813949c49addbbf630361/qa/tasks 2024-09-13T15:33:15.880 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T15:33:15.881 INFO:teuthology.task.internal:Checking packages... 2024-09-13T15:33:15.901 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '50bbe7f395997dcd871813949c49addbbf630361' 2024-09-13T15:33:15.902 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T15:33:15.902 INFO:teuthology.packaging:ref: None 2024-09-13T15:33:15.902 INFO:teuthology.packaging:tag: None 2024-09-13T15:33:15.902 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240910.095512-reef 2024-09-13T15:33:15.902 INFO:teuthology.packaging:sha1: 50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T15:33:15.902 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240910.095512-reef 2024-09-13T15:33:16.052 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-850.g50bbe7f3 2024-09-13T15:33:16.054 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T15:33:16.062 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T15:33:16.062 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T15:33:16.077 INFO:teuthology.task.internal:Saving configuration 2024-09-13T15:33:16.093 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T15:33:16.102 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T15:33:16.135 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903753', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 15:28:15.993890', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGYfV3u+lofsJ2Y5eqDVRbxn2FHE5sUjgnZA8eB2AzM+gitm8GrJxaSI77j03KVieidlNDdKexVafxYub7tmiug='} 2024-09-13T15:33:16.136 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T15:33:16.141 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2024-09-13T15:33:16.142 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T15:33:16.187 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb14d3cfbe0>, signals=[15]) 2024-09-13T15:33:16.187 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T15:33:16.196 INFO:teuthology.task.internal:Opening connections... 2024-09-13T15:33:16.196 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T15:33:16.197 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T15:33:16.280 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T15:33:16.288 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2024-09-13T15:33:16.358 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2024-09-13T15:33:16.359 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2024-09-13T15:33:16.417 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2024-09-13T15:33:16.418 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="9" 2024-09-13T15:33:16.418 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2024-09-13T15:33:16.418 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2024-09-13T15:33:16.418 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="9" 2024-09-13T15:33:16.418 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el9" 2024-09-13T15:33:16.418 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T15:33:16.418 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2024-09-13T15:33:16.418 INFO:teuthology.orchestra.run.smithi070.stdout:LOGO="fedora-logo-icon" 2024-09-13T15:33:16.418 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T15:33:16.418 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2024-09-13T15:33:16.418 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T15:33:16.418 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T15:33:16.419 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T15:33:16.419 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2024-09-13T15:33:16.477 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T15:33:16.488 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T15:33:16.494 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T15:33:16.494 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2024-09-13T15:33:16.511 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T15:33:16.519 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T15:33:16.519 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2024-09-13T15:33:16.805 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T15:33:16.865 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T15:33:16.881 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T15:33:16.881 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T15:33:16.881 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T15:33:16.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T15:33:16.899 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-09-13T15:33:16.956 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-503.el9.x86_64 2024-09-13T15:33:16.957 INFO:teuthology.task.kernel:Running kernel on smithi070: 5.14.0-503.el9.x86_64 2024-09-13T15:33:16.957 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum install -y kernel 2024-09-13T15:33:19.483 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-13T15:33:22.617 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-13T15:33:28.067 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.5 MB 00:01 2024-09-13T15:33:29.709 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-13T15:33:31.254 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-13T15:33:36.710 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-13T15:33:38.741 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T15:33:38.741 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T15:33:38.840 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:33:38.856 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:33:38.856 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-09-13T15:33:38.856 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:33:38.857 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-13T15:33:38.857 INFO:teuthology.orchestra.run.smithi070.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T15:33:38.857 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-13T15:33:38.857 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T15:33:38.857 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T15:33:38.857 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T15:33:38.857 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:33:38.857 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-13T15:33:38.857 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:33:38.857 INFO:teuthology.orchestra.run.smithi070.stdout:Install 4 Packages 2024-09-13T15:33:38.858 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:33:38.858 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 79 M 2024-09-13T15:33:38.858 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 126 M 2024-09-13T15:33:38.858 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-13T15:33:39.278 INFO:teuthology.orchestra.run.smithi070.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 159 kB/s | 23 kB 00:00 2024-09-13T15:33:39.853 INFO:teuthology.orchestra.run.smithi070.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-13T15:33:40.403 INFO:teuthology.orchestra.run.smithi070.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 27 MB/s | 35 MB 00:01 2024-09-13T15:33:41.254 INFO:teuthology.orchestra.run.smithi070.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 14 MB/s | 29 MB 00:01 2024-09-13T15:33:41.254 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-13T15:33:41.254 INFO:teuthology.orchestra.run.smithi070.stdout:Total 33 MB/s | 79 MB 00:02 2024-09-13T15:33:41.938 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-13T15:33:42.295 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-13T15:33:42.295 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-13T15:33:43.247 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-13T15:33:43.247 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-13T15:33:45.714 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-13T15:33:46.369 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T15:33:46.445 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T15:33:49.243 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T15:33:49.382 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T15:33:54.193 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T15:33:54.231 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T15:33:59.089 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T15:34:37.452 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T15:34:37.492 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T15:34:37.771 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T15:34:37.772 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T15:34:37.772 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T15:34:37.772 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T15:34:38.590 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T15:34:38.590 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:34:38.590 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-13T15:34:38.590 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T15:34:38.591 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T15:34:38.591 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T15:34:38.591 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T15:34:38.591 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:34:38.591 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:34:38.913 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T15:34:39.005 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T15:34:39.006 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T15:34:39.006 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T15:34:39.006 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T15:34:39.006 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T15:34:39.006 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T15:34:39.006 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi070.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T15:34:39.007 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T15:34:39.065 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T15:34:39.065 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T15:34:39.066 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -qi grub2-tools 2024-09-13T15:34:39.105 INFO:teuthology.orchestra.run.smithi070.stdout:Name : grub2-tools 2024-09-13T15:34:39.105 INFO:teuthology.orchestra.run.smithi070.stdout:Epoch : 1 2024-09-13T15:34:39.105 INFO:teuthology.orchestra.run.smithi070.stdout:Version : 2.06 2024-09-13T15:34:39.105 INFO:teuthology.orchestra.run.smithi070.stdout:Release : 82.el9 2024-09-13T15:34:39.105 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: x86_64 2024-09-13T15:34:39.105 INFO:teuthology.orchestra.run.smithi070.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T15:34:39.105 INFO:teuthology.orchestra.run.smithi070.stdout:Group : Unspecified 2024-09-13T15:34:39.106 INFO:teuthology.orchestra.run.smithi070.stdout:Size : 8274670 2024-09-13T15:34:39.106 INFO:teuthology.orchestra.run.smithi070.stdout:License : GPLv3+ 2024-09-13T15:34:39.106 INFO:teuthology.orchestra.run.smithi070.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T15:34:39.106 INFO:teuthology.orchestra.run.smithi070.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T15:34:39.106 INFO:teuthology.orchestra.run.smithi070.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T15:34:39.106 INFO:teuthology.orchestra.run.smithi070.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T15:34:39.106 INFO:teuthology.orchestra.run.smithi070.stdout:Packager : builder@centos.org 2024-09-13T15:34:39.106 INFO:teuthology.orchestra.run.smithi070.stdout:Vendor : CentOS 2024-09-13T15:34:39.106 INFO:teuthology.orchestra.run.smithi070.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T15:34:39.106 INFO:teuthology.orchestra.run.smithi070.stdout:Summary : Support tools for GRUB. 2024-09-13T15:34:39.106 INFO:teuthology.orchestra.run.smithi070.stdout:Description : 2024-09-13T15:34:39.106 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:34:39.106 INFO:teuthology.orchestra.run.smithi070.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T15:34:39.107 INFO:teuthology.orchestra.run.smithi070.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T15:34:39.107 INFO:teuthology.orchestra.run.smithi070.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T15:34:39.107 INFO:teuthology.orchestra.run.smithi070.stdout:hardware devices. 2024-09-13T15:34:39.107 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:34:39.107 INFO:teuthology.orchestra.run.smithi070.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T15:34:39.108 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T15:34:39.108 INFO:teuthology.task.kernel:Updating grub on smithi070 to boot 5.14.0-505.el9.x86_64 2024-09-13T15:34:39.108 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T15:34:39.757 INFO:teuthology.orchestra.run.smithi070.stderr:Generating grub configuration file ... 2024-09-13T15:34:41.110 INFO:teuthology.orchestra.run.smithi070.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T15:34:41.138 INFO:teuthology.orchestra.run.smithi070.stderr:done 2024-09-13T15:34:41.141 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-13T15:34:41.157 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.NFomynUgwi 2024-09-13T15:34:41.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.NFomynUgwi 2024-09-13T15:34:41.225 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.NFomynUgwi 2024-09-13T15:34:41.370 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.NFomynUgwi is 6KB 2024-09-13T15:34:41.422 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.NFomynUgwi 2024-09-13T15:34:41.438 DEBUG:teuthology.orchestra.run.smithi070:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T15:34:41.505 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T15:34:41.505 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T15:34:41.505 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T15:34:41.505 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T15:34:41.507 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T15:34:41.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo shutdown -r now 2024-09-13T15:35:11.709 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T15:35:11.710 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-13T15:35:11.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T15:35:30.109 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-09-13T15:35:39.117 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-13T15:35:39.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T15:35:42.209 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2024-09-13T15:35:54.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-13T15:35:54.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T15:36:26.037 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-09-13T15:36:26.468 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-13T15:36:26.469 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-13T15:36:26.469 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-09-13T15:36:26.483 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-505.el9.x86_64 2024-09-13T15:36:26.484 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T15:36:26.484 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T15:36:26.484 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T15:36:27.484 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T15:36:27.485 DEBUG:teuthology.orchestra.run.smithi070:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T15:36:27.700 INFO:teuthology.orchestra.run.smithi070.stdout:ttyS1 2024-09-13T15:36:27.727 DEBUG:teuthology.parallel:result is None 2024-09-13T15:36:27.728 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T15:36:27.738 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T15:36:27.738 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T15:36:27.753 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T15:36:27.763 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T15:36:27.775 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T15:36:27.776 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T15:36:27.822 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T15:36:27.833 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T15:36:27.834 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T15:36:28.207 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T15:36:28.226 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T15:36:28.229 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T15:36:28.238 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T15:36:28.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T15:36:28.297 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T15:36:28.310 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T15:36:28.310 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T15:36:28.342 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T15:36:28.461 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T15:36:28.526 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T15:36:28.607 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T15:36:28.689 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:36:28.689 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T15:36:28.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2024-09-13T15:36:28.881 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T15:36:29.373 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T15:36:29.384 INFO:teuthology.task.internal:Starting timer... 2024-09-13T15:36:29.384 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T15:36:29.401 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T15:36:29.416 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-13T15:36:29.416 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2024-09-13T15:36:29.482 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2024-09-13T15:36:29.484 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T15:36:29.484 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2024-09-13T15:36:29.502 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2024-09-13T15:36:29.502 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi070.front.sepia.ceph.com': 'permissive'} 2024-09-13T15:36:29.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T15:36:29.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T15:36:29.573 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T15:36:29.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2024-09-13T15:36:29.682 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T15:36:29.694 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T15:36:29.705 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T15:36:29.706 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi070.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T15:40:14.738 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi070.front.sepia.ceph.com')] 2024-09-13T15:40:14.739 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-13T15:40:14.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T15:40:14.826 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-09-13T15:40:14.866 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-13T15:40:14.867 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T15:40:14.878 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T15:40:14.879 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T15:40:14.879 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T15:40:14.936 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T15:40:14.950 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T15:40:14.973 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2024-09-13T15:40:14.984 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2024-09-13T15:40:14.999 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T15:40:15.013 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T15:40:15.071 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-09-13T15:40:15.076 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T15:40:15.076 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-09-13T15:40:15.076 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T15:40:15.076 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T15:40:15.076 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T15:40:15.076 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T15:40:15.082 INFO:teuthology.run_tasks:Running task install... 2024-09-13T15:40:15.090 DEBUG:teuthology.task.install:project ceph 2024-09-13T15:40:15.090 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}} 2024-09-13T15:40:15.091 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'} 2024-09-13T15:40:15.091 INFO:teuthology.task.install:Using flavor: default 2024-09-13T15:40:15.097 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T15:40:15.097 INFO:teuthology.task.install:extra packages: ['librados2'] 2024-09-13T15:40:15.098 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '50bbe7f395997dcd871813949c49addbbf630361', 'tag': None, 'wait_for_package': False} 2024-09-13T15:40:15.098 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T15:40:15.099 INFO:teuthology.packaging:ref: None 2024-09-13T15:40:15.099 INFO:teuthology.packaging:tag: None 2024-09-13T15:40:15.099 INFO:teuthology.packaging:branch: quincy 2024-09-13T15:40:15.099 INFO:teuthology.packaging:sha1: 50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T15:40:15.099 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-09-13T15:40:15.314 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/bbdadfa31f8fb9afaa389358d138cac49bd69e56/centos/9/flavors/default/ 2024-09-13T15:40:15.314 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1627.gbbdadfa3 2024-09-13T15:40:15.436 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/bbdadfa31f8fb9afaa389358d138cac49bd69e56/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/bbdadfa31f8fb9afaa389358d138cac49bd69e56/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/bbdadfa31f8fb9afaa389358d138cac49bd69e56/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T15:40:15.436 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:40:15.436 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T15:40:15.462 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-immutable-object-cache, ceph-mgr, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, librados2 on remote rpm x86_64 2024-09-13T15:40:15.463 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T15:40:15.463 INFO:teuthology.packaging:ref: None 2024-09-13T15:40:15.463 INFO:teuthology.packaging:tag: None 2024-09-13T15:40:15.463 INFO:teuthology.packaging:branch: quincy 2024-09-13T15:40:15.463 INFO:teuthology.packaging:sha1: 50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T15:40:15.463 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T15:40:15.530 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T15:40:15.605 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T15:40:15.670 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2024-09-13T15:40:15.671 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2024-09-13T15:40:15.913 INFO:teuthology.orchestra.run.smithi070.stdout:45 files removed 2024-09-13T15:40:15.939 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw 2024-09-13T15:40:16.478 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 305 kB/s | 77 kB 00:00 2024-09-13T15:40:16.745 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-13T15:40:17.003 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 8.0 kB/s | 1.8 kB 00:00 2024-09-13T15:40:18.803 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-13T15:40:20.813 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-13T15:40:25.371 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.5 MB 00:00 2024-09-13T15:40:26.863 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 118 kB/s | 19 kB 00:00 2024-09-13T15:40:28.076 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-13T15:40:33.590 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-13T15:40:35.932 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:40:35.946 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:40:35.946 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-13T15:40:35.946 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:40:35.946 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-13T15:40:35.946 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 14 M 2024-09-13T15:40:35.946 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2024-09-13T15:40:35.946 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 3.1 M 2024-09-13T15:40:35.946 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 2.9 M 2024-09-13T15:40:35.947 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-13T15:40:35.947 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 5.0 M 2024-09-13T15:40:35.947 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 22 M 2024-09-13T15:40:35.947 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 25 k 2024-09-13T15:40:35.947 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T15:40:35.947 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 637 k 2024-09-13T15:40:35.947 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T15:40:35.947 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T15:40:35.947 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T15:40:35.947 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 457 k 2024-09-13T15:40:35.947 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T15:40:35.947 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 4.9 M 2024-09-13T15:40:35.948 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T15:40:35.948 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T15:40:35.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 45 k 2024-09-13T15:40:35.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 120 k 2024-09-13T15:40:35.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 156 k 2024-09-13T15:40:35.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T15:40:35.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 320 k 2024-09-13T15:40:35.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 294 k 2024-09-13T15:40:35.948 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 99 k 2024-09-13T15:40:35.948 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T15:40:35.948 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:40:35.948 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-13T15:40:35.948 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:40:35.949 INFO:teuthology.orchestra.run.smithi070.stdout:Install 22 Packages 2024-09-13T15:40:35.949 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 2 Packages 2024-09-13T15:40:35.949 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:40:35.949 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 58 M 2024-09-13T15:40:35.949 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-13T15:40:37.007 INFO:teuthology.orchestra.run.smithi070.stdout:(1/24): ceph-base-17.2.7-1627.gbbdadfa3.el9.x86 6.6 MB/s | 5.0 MB 00:00 2024-09-13T15:40:37.057 INFO:teuthology.orchestra.run.smithi070.stdout:(2/24): ceph-selinux-17.2.7-1627.gbbdadfa3.el9. 504 kB/s | 25 kB 00:00 2024-09-13T15:40:37.133 INFO:teuthology.orchestra.run.smithi070.stdout:(3/24): libcephfs2-17.2.7-1627.gbbdadfa3.el9.x8 8.3 MB/s | 637 kB 00:00 2024-09-13T15:40:37.216 INFO:teuthology.orchestra.run.smithi070.stdout:(4/24): libradosstriper1-17.2.7-1627.gbbdadfa3. 5.4 MB/s | 457 kB 00:00 2024-09-13T15:40:37.608 INFO:teuthology.orchestra.run.smithi070.stdout:(5/24): ceph-radosgw-17.2.7-1627.gbbdadfa3.el9. 11 MB/s | 14 MB 00:01 2024-09-13T15:40:37.683 INFO:teuthology.orchestra.run.smithi070.stdout:(6/24): librgw2-17.2.7-1627.gbbdadfa3.el9.x86_6 11 MB/s | 4.9 MB 00:00 2024-09-13T15:40:37.708 INFO:teuthology.orchestra.run.smithi070.stdout:(7/24): python3-ceph-argparse-17.2.7-1627.gbbda 452 kB/s | 45 kB 00:00 2024-09-13T15:40:37.725 INFO:teuthology.orchestra.run.smithi070.stdout:(8/24): python3-ceph-common-17.2.7-1627.gbbdadf 2.8 MB/s | 120 kB 00:00 2024-09-13T15:40:37.759 INFO:teuthology.orchestra.run.smithi070.stdout:(9/24): python3-cephfs-17.2.7-1627.gbbdadfa3.el 3.0 MB/s | 156 kB 00:00 2024-09-13T15:40:37.784 INFO:teuthology.orchestra.run.smithi070.stdout:(10/24): python3-rados-17.2.7-1627.gbbdadfa3.el 5.3 MB/s | 320 kB 00:00 2024-09-13T15:40:37.810 INFO:teuthology.orchestra.run.smithi070.stdout:(11/24): python3-rbd-17.2.7-1627.gbbdadfa3.el9. 5.7 MB/s | 294 kB 00:00 2024-09-13T15:40:37.826 INFO:teuthology.orchestra.run.smithi070.stdout:(12/24): python3-rgw-17.2.7-1627.gbbdadfa3.el9. 2.3 MB/s | 99 kB 00:00 2024-09-13T15:40:38.052 INFO:teuthology.orchestra.run.smithi070.stdout:(13/24): ceph-common-17.2.7-1627.gbbdadfa3.el9. 13 MB/s | 22 MB 00:01 2024-09-13T15:40:38.127 INFO:teuthology.orchestra.run.smithi070.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 151 kB/s | 45 kB 00:00 2024-09-13T15:40:38.186 INFO:teuthology.orchestra.run.smithi070.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 426 kB/s | 160 kB 00:00 2024-09-13T15:40:38.236 INFO:teuthology.orchestra.run.smithi070.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 850 kB/s | 42 kB 00:00 2024-09-13T15:40:38.261 INFO:teuthology.orchestra.run.smithi070.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-13T15:40:38.297 INFO:teuthology.orchestra.run.smithi070.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 662 kB 00:00 2024-09-13T15:40:38.347 INFO:teuthology.orchestra.run.smithi070.stdout:(19/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.7 MB/s | 308 kB 00:00 2024-09-13T15:40:38.364 INFO:teuthology.orchestra.run.smithi070.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 513 kB/s | 49 kB 00:00 2024-09-13T15:40:38.389 INFO:teuthology.orchestra.run.smithi070.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 816 kB/s | 67 kB 00:00 2024-09-13T15:40:38.481 INFO:teuthology.orchestra.run.smithi070.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-13T15:40:38.573 INFO:teuthology.orchestra.run.smithi070.stdout:(23/24): librados2-17.2.7-1627.gbbdadfa3.el9.x8 15 MB/s | 3.1 MB 00:00 2024-09-13T15:40:38.623 INFO:teuthology.orchestra.run.smithi070.stdout:(24/24): librbd1-17.2.7-1627.gbbdadfa3.el9.x86_ 13 MB/s | 2.9 MB 00:00 2024-09-13T15:40:38.628 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-13T15:40:38.629 INFO:teuthology.orchestra.run.smithi070.stdout:Total 22 MB/s | 58 MB 00:02 2024-09-13T15:40:38.693 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-13T15:40:38.727 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-13T15:40:38.727 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-13T15:40:39.379 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-13T15:40:39.380 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-13T15:40:40.592 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-13T15:40:40.927 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-13T15:40:41.034 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2/26 2024-09-13T15:40:41.127 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2/26 2024-09-13T15:40:41.204 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:17.2.7-1627.gbbdadfa3.el9.x86_64 3/26 2024-09-13T15:40:41.244 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 4/26 2024-09-13T15:40:41.331 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 4/26 2024-09-13T15:40:41.377 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-13T15:40:41.415 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-13T15:40:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-13T15:40:41.880 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 8/26 2024-09-13T15:40:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 8/26 2024-09-13T15:40:41.971 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-13T15:40:42.014 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:17.2.7-1627.gbbdadfa3.el9. 10/26 2024-09-13T15:40:42.051 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:17.2.7-1627.gbbdadfa3.el9.x86_64 11/26 2024-09-13T15:40:42.113 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 12/26 2024-09-13T15:40:42.141 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86_6 13/26 2024-09-13T15:40:42.190 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86_6 13/26 2024-09-13T15:40:42.245 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-13T15:40:42.274 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-13T15:40:42.307 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-13T15:40:42.599 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-13T15:40:42.645 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 18/26 2024-09-13T15:40:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 18/26 2024-09-13T15:40:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 19/26 2024-09-13T15:40:42.839 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x8 20/26 2024-09-13T15:40:45.329 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 21/26 2024-09-13T15:40:45.372 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 21/26 2024-09-13T15:40:45.886 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 21/26 2024-09-13T15:40:45.922 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 22/26 2024-09-13T15:40:46.020 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 22/26 2024-09-13T15:40:46.021 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T15:40:46.021 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T15:40:46.021 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:40:46.057 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 23/26 2024-09-13T15:40:55.836 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 23/26 2024-09-13T15:40:55.836 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2024-09-13T15:40:55.836 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2024-09-13T15:40:55.836 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2024-09-13T15:40:55.836 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2024-09-13T15:40:55.837 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2024-09-13T15:40:55.837 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /root 2024-09-13T15:40:55.837 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2024-09-13T15:40:55.837 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:40:57.161 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 24/26 2024-09-13T15:40:57.238 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 24/26 2024-09-13T15:40:57.238 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T15:40:57.238 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T15:40:57.238 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T15:40:57.239 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T15:40:57.239 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:40:57.240 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-13T15:40:57.330 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-13T15:40:57.331 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-13T15:40:58.585 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-13T15:40:58.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/26 2024-09-13T15:40:58.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2/26 2024-09-13T15:40:58.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 3/26 2024-09-13T15:40:58.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 4/26 2024-09-13T15:40:58.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 5/26 2024-09-13T15:40:58.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86_6 6/26 2024-09-13T15:40:58.594 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 7/26 2024-09-13T15:40:58.594 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1627.gbbdadfa3.el9. 8/26 2024-09-13T15:40:58.594 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x8 9/26 2024-09-13T15:40:58.594 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:17.2.7-1627.gbbdadfa3.el9.x86_64 10/26 2024-09-13T15:40:58.594 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:17.2.7-1627.gbbdadfa3.el9.x86_64 11/26 2024-09-13T15:40:58.594 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 12/26 2024-09-13T15:40:58.594 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 13/26 2024-09-13T15:40:58.594 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-13T15:40:58.594 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-13T15:40:58.594 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-13T15:40:58.594 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-13T15:40:58.595 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-13T15:40:58.595 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-13T15:40:58.595 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-13T15:40:58.595 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-13T15:40:58.595 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-13T15:40:58.595 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 23/26 2024-09-13T15:40:58.595 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-13T15:40:58.595 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 25/26 2024-09-13T15:40:59.513 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-13T15:40:59.514 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:40:59.514 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2024-09-13T15:40:59.514 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:40:59.514 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:40:59.514 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-13T15:40:59.514 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:40:59.514 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:40:59.514 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:40:59.514 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:40:59.514 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T15:40:59.514 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:40:59.514 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T15:40:59.515 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T15:40:59.515 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T15:40:59.515 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:40:59.515 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T15:40:59.515 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:40:59.515 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T15:40:59.515 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T15:40:59.515 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:40:59.515 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:40:59.515 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:40:59.515 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T15:40:59.515 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:40:59.515 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:40:59.516 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:40:59.516 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T15:40:59.516 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:40:59.516 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:40:59.874 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-test 2024-09-13T15:41:00.442 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:27 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:00.571 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:00.572 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:00.572 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-13T15:41:00.572 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:00.572 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-13T15:41:00.572 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 48 M 2024-09-13T15:41:00.573 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-13T15:41:00.573 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 163 k 2024-09-13T15:41:00.573 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T15:41:00.573 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T15:41:00.573 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:00.573 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-13T15:41:00.573 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:00.573 INFO:teuthology.orchestra.run.smithi070.stdout:Install 4 Packages 2024-09-13T15:41:00.573 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:00.574 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 48 M 2024-09-13T15:41:00.574 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 193 M 2024-09-13T15:41:00.574 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-13T15:41:00.827 INFO:teuthology.orchestra.run.smithi070.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.3 MB/s | 304 kB 00:00 2024-09-13T15:41:00.869 INFO:teuthology.orchestra.run.smithi070.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-13T15:41:00.928 INFO:teuthology.orchestra.run.smithi070.stdout:(3/4): libcephsqlite-17.2.7-1627.gbbdadfa3.el9. 703 kB/s | 163 kB 00:00 2024-09-13T15:41:02.253 INFO:teuthology.orchestra.run.smithi070.stdout:(4/4): ceph-test-17.2.7-1627.gbbdadfa3.el9.x86_ 31 MB/s | 48 MB 00:01 2024-09-13T15:41:02.254 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-13T15:41:02.255 INFO:teuthology.orchestra.run.smithi070.stdout:Total 29 MB/s | 48 MB 00:01 2024-09-13T15:41:02.282 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-13T15:41:02.300 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-13T15:41:02.301 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-13T15:41:02.663 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-13T15:41:02.664 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-13T15:41:03.178 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-13T15:41:03.263 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-13T15:41:03.309 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-13T15:41:03.338 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 3/4 2024-09-13T15:41:07.706 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 3/4 2024-09-13T15:41:07.777 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:17.2.7-1627.gbbdadfa3.el9.x86_64 4/4 2024-09-13T15:41:09.974 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-test-2:17.2.7-1627.gbbdadfa3.el9.x86_64 4/4 2024-09-13T15:41:09.974 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/4 2024-09-13T15:41:09.974 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2/4 2024-09-13T15:41:09.974 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-13T15:41:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-13T15:41:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:10.298 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-13T15:41:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:41:10.298 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:41:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T15:41:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T15:41:10.299 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:10.299 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:10.572 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph 2024-09-13T15:41:11.146 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:38 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:11.282 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:11.283 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:11.283 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-13T15:41:11.283 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:11.283 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-13T15:41:11.284 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 6.5 k 2024-09-13T15:41:11.284 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-13T15:41:11.284 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 2.1 M 2024-09-13T15:41:11.284 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 1.4 M 2024-09-13T15:41:11.284 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1627.gbbdadfa3.el9 ceph-noarch 240 k 2024-09-13T15:41:11.284 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 3.9 M 2024-09-13T15:41:11.284 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 15 M 2024-09-13T15:41:11.284 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T15:41:11.284 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T15:41:11.284 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T15:41:11.284 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T15:41:11.284 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T15:41:11.284 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T15:41:11.285 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T15:41:11.285 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections 2024-09-13T15:41:11.285 INFO:teuthology.orchestra.run.smithi070.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-13T15:41:11.285 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T15:41:11.285 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T15:41:11.285 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T15:41:11.285 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T15:41:11.285 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T15:41:11.285 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T15:41:11.285 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T15:41:11.285 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T15:41:11.285 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T15:41:11.285 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T15:41:11.285 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T15:41:11.286 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T15:41:11.286 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T15:41:11.286 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T15:41:11.286 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T15:41:11.286 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T15:41:11.286 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T15:41:11.286 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T15:41:11.286 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T15:41:11.286 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T15:41:11.286 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:11.286 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-13T15:41:11.286 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:11.286 INFO:teuthology.orchestra.run.smithi070.stdout:Install 34 Packages 2024-09-13T15:41:11.287 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:11.288 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 27 M 2024-09-13T15:41:11.288 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 96 M 2024-09-13T15:41:11.288 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-13T15:41:11.503 INFO:teuthology.orchestra.run.smithi070.stdout:(1/34): ceph-17.2.7-1627.gbbdadfa3.el9.x86_64.r 54 kB/s | 6.5 kB 00:00 2024-09-13T15:41:11.678 INFO:teuthology.orchestra.run.smithi070.stdout:(2/34): ceph-mgr-17.2.7-1627.gbbdadfa3.el9.x86_ 4.8 MB/s | 1.4 MB 00:00 2024-09-13T15:41:11.737 INFO:teuthology.orchestra.run.smithi070.stdout:(3/34): ceph-mds-17.2.7-1627.gbbdadfa3.el9.x86_ 5.9 MB/s | 2.1 MB 00:00 2024-09-13T15:41:11.821 INFO:teuthology.orchestra.run.smithi070.stdout:(4/34): ceph-mon-17.2.7-1627.gbbdadfa3.el9.x86_ 12 MB/s | 3.9 MB 00:00 2024-09-13T15:41:11.846 INFO:teuthology.orchestra.run.smithi070.stdout:(5/34): ceph-mgr-modules-core-17.2.7-1627.gbbda 2.2 MB/s | 240 kB 00:00 2024-09-13T15:41:12.171 INFO:teuthology.orchestra.run.smithi070.stdout:(6/34): ceph-osd-17.2.7-1627.gbbdadfa3.el9.x86_ 31 MB/s | 15 MB 00:00 2024-09-13T15:41:12.805 INFO:teuthology.orchestra.run.smithi070.stdout:(7/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 257 kB/s | 253 kB 00:00 2024-09-13T15:41:12.838 INFO:teuthology.orchestra.run.smithi070.stdout:(8/34): python3-ply-3.11-14.el9.noarch.rpm 163 kB/s | 106 kB 00:00 2024-09-13T15:41:12.930 INFO:teuthology.orchestra.run.smithi070.stdout:(9/34): python3-cryptography-36.0.1-4.el9.x86_6 1.1 MB/s | 1.2 MB 00:01 2024-09-13T15:41:13.497 INFO:teuthology.orchestra.run.smithi070.stdout:(10/34): python3-pycparser-2.20-6.el9.noarch.rp 195 kB/s | 135 kB 00:00 2024-09-13T15:41:13.539 INFO:teuthology.orchestra.run.smithi070.stdout:(11/34): python3-requests-2.25.1-8.el9.noarch.r 178 kB/s | 125 kB 00:00 2024-09-13T15:41:13.631 INFO:teuthology.orchestra.run.smithi070.stdout:(12/34): python3-markupsafe-1.1.1-12.el9.x86_64 382 kB/s | 35 kB 00:00 2024-09-13T15:41:13.656 INFO:teuthology.orchestra.run.smithi070.stdout:(13/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-13T15:41:13.681 INFO:teuthology.orchestra.run.smithi070.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-13T15:41:13.707 INFO:teuthology.orchestra.run.smithi070.stdout:(15/34): python3-urllib3-1.26.5-6.el9.noarch.rp 277 kB/s | 215 kB 00:00 2024-09-13T15:41:13.732 INFO:teuthology.orchestra.run.smithi070.stdout:(16/34): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-09-13T15:41:13.757 INFO:teuthology.orchestra.run.smithi070.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 560 kB/s | 42 kB 00:00 2024-09-13T15:41:13.816 INFO:teuthology.orchestra.run.smithi070.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 400 kB/s | 43 kB 00:00 2024-09-13T15:41:13.842 INFO:teuthology.orchestra.run.smithi070.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 467 kB/s | 11 kB 00:00 2024-09-13T15:41:13.866 INFO:teuthology.orchestra.run.smithi070.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-13T15:41:13.883 INFO:teuthology.orchestra.run.smithi070.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 421 kB/s | 18 kB 00:00 2024-09-13T15:41:13.900 INFO:teuthology.orchestra.run.smithi070.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 693 kB/s | 23 kB 00:00 2024-09-13T15:41:13.917 INFO:teuthology.orchestra.run.smithi070.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 581 kB/s | 19 kB 00:00 2024-09-13T15:41:13.934 INFO:teuthology.orchestra.run.smithi070.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 587 kB/s | 20 kB 00:00 2024-09-13T15:41:13.959 INFO:teuthology.orchestra.run.smithi070.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 1.1 MB/s | 46 kB 00:00 2024-09-13T15:41:13.985 INFO:teuthology.orchestra.run.smithi070.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-13T15:41:14.010 INFO:teuthology.orchestra.run.smithi070.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-13T15:41:14.035 INFO:teuthology.orchestra.run.smithi070.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-13T15:41:14.061 INFO:teuthology.orchestra.run.smithi070.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-13T15:41:14.086 INFO:teuthology.orchestra.run.smithi070.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-09-13T15:41:14.111 INFO:teuthology.orchestra.run.smithi070.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-13T15:41:14.136 INFO:teuthology.orchestra.run.smithi070.stdout:(32/34): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-13T15:41:14.162 INFO:teuthology.orchestra.run.smithi070.stdout:(33/34): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-09-13T15:41:14.187 INFO:teuthology.orchestra.run.smithi070.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-13T15:41:14.193 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-13T15:41:14.193 INFO:teuthology.orchestra.run.smithi070.stdout:Total 9.4 MB/s | 27 MB 00:02 2024-09-13T15:41:14.324 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-13T15:41:14.362 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-13T15:41:14.362 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-13T15:41:14.743 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-13T15:41:14.744 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-13T15:41:16.008 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-13T15:41:16.073 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-13T15:41:16.106 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-13T15:41:16.139 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-13T15:41:16.229 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-13T15:41:16.308 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-13T15:41:16.349 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-13T15:41:16.384 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-13T15:41:16.414 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-13T15:41:16.449 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-13T15:41:16.482 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-13T15:41:16.522 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-13T15:41:16.569 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-13T15:41:16.604 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-13T15:41:16.652 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-13T15:41:16.684 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-13T15:41:16.756 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-13T15:41:16.894 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-13T15:41:16.975 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-13T15:41:17.028 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-13T15:41:17.067 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-13T15:41:17.116 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-13T15:41:17.171 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-13T15:41:17.356 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-13T15:41:17.405 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-13T15:41:17.500 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-13T15:41:17.587 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-13T15:41:17.620 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-13T15:41:17.735 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-13T15:41:17.891 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1627.gbbdadfa3.el9. 29/34 2024-09-13T15:41:17.923 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 30/34 2024-09-13T15:41:17.949 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 30/34 2024-09-13T15:41:17.949 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T15:41:17.949 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T15:41:17.949 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T15:41:17.949 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T15:41:17.949 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:19.265 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 31/34 2024-09-13T15:41:19.292 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 31/34 2024-09-13T15:41:19.292 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T15:41:19.292 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T15:41:19.292 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T15:41:19.292 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T15:41:19.293 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:19.666 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 32/34 2024-09-13T15:41:19.690 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 32/34 2024-09-13T15:41:19.690 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T15:41:19.690 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T15:41:19.690 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T15:41:19.690 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T15:41:19.690 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:19.891 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 33/34 2024-09-13T15:41:19.915 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 33/34 2024-09-13T15:41:19.915 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T15:41:19.915 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T15:41:19.915 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T15:41:19.916 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T15:41:19.916 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:19.980 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:17.2.7-1627.gbbdadfa3.el9.x86_64 34/34 2024-09-13T15:41:21.696 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-2:17.2.7-1627.gbbdadfa3.el9.x86_64 34/34 2024-09-13T15:41:21.696 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/34 2024-09-13T15:41:21.696 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2/34 2024-09-13T15:41:21.696 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 3/34 2024-09-13T15:41:21.696 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 4/34 2024-09-13T15:41:21.697 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 5/34 2024-09-13T15:41:21.697 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1627.gbbdadfa3.el9. 6/34 2024-09-13T15:41:21.697 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-13T15:41:21.697 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-13T15:41:21.697 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-13T15:41:21.697 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-13T15:41:21.697 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-13T15:41:21.697 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-13T15:41:21.697 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-13T15:41:21.697 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-13T15:41:21.697 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-13T15:41:21.698 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-13T15:41:21.698 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-13T15:41:21.698 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-13T15:41:21.698 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-13T15:41:21.698 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-13T15:41:21.698 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-13T15:41:21.698 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-13T15:41:21.698 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-13T15:41:21.698 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-13T15:41:21.698 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-13T15:41:21.699 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-13T15:41:21.699 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-13T15:41:21.699 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-13T15:41:21.699 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-13T15:41:21.700 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-13T15:41:21.700 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-13T15:41:21.700 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-13T15:41:21.700 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-13T15:41:22.587 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-13T15:41:22.588 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:22.588 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-13T15:41:22.588 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:41:22.588 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:41:22.588 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:41:22.589 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:17.2.7-1627.gbbdadfa3.el9.noarch 2024-09-13T15:41:22.589 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:41:22.589 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:41:22.589 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T15:41:22.589 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T15:41:22.590 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T15:41:22.590 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T15:41:22.590 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T15:41:22.590 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T15:41:22.590 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T15:41:22.590 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T15:41:22.590 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T15:41:22.591 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T15:41:22.591 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T15:41:22.591 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T15:41:22.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T15:41:22.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T15:41:22.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T15:41:22.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T15:41:22.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T15:41:22.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T15:41:22.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T15:41:22.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T15:41:22.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T15:41:22.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T15:41:22.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T15:41:22.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T15:41:22.592 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T15:41:22.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T15:41:22.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T15:41:22.593 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T15:41:22.593 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:22.593 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:22.943 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-base 2024-09-13T15:41:23.513 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:50 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:23.604 INFO:teuthology.orchestra.run.smithi070.stdout:Package ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 is already installed. 2024-09-13T15:41:23.645 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:23.646 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-13T15:41:23.648 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:23.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-immutable-object-cache 2024-09-13T15:41:24.264 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:51 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:24.394 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:24.395 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:24.395 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-13T15:41:24.395 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:24.395 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-13T15:41:24.395 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 137 k 2024-09-13T15:41:24.396 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:24.396 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-13T15:41:24.396 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:24.396 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-13T15:41:24.396 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:24.396 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 137 k 2024-09-13T15:41:24.396 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 379 k 2024-09-13T15:41:24.396 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-13T15:41:24.584 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-immutable-object-cache-17.2.7-1627.gbbdadf 733 kB/s | 137 kB 00:00 2024-09-13T15:41:24.584 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-13T15:41:24.585 INFO:teuthology.orchestra.run.smithi070.stdout:Total 727 kB/s | 137 kB 00:00 2024-09-13T15:41:24.585 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-13T15:41:24.592 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-13T15:41:24.592 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-13T15:41:24.648 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-13T15:41:24.648 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-13T15:41:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-13T15:41:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1627.gbbdadfa3. 1/1 2024-09-13T15:41:24.860 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1627.gbbdadfa3. 1/1 2024-09-13T15:41:24.860 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T15:41:24.860 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T15:41:24.860 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:25.850 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1627.gbbdadfa3. 1/1 2024-09-13T15:41:25.850 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:25.850 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-13T15:41:25.850 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:41:25.850 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:25.850 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:26.047 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr 2024-09-13T15:41:26.624 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:53 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:26.716 INFO:teuthology.orchestra.run.smithi070.stdout:Package ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 is already installed. 2024-09-13T15:41:26.758 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:26.759 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-13T15:41:26.759 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:26.821 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-fuse 2024-09-13T15:41:27.386 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:54 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:27.515 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:27.516 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-13T15:41:27.516 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:27.516 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-13T15:41:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 813 k 2024-09-13T15:41:27.516 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-13T15:41:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T15:41:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:27.516 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-13T15:41:27.516 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:27.516 INFO:teuthology.orchestra.run.smithi070.stdout:Install 2 Packages 2024-09-13T15:41:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:27.517 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 893 k 2024-09-13T15:41:27.517 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 2.6 M 2024-09-13T15:41:27.517 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-13T15:41:27.788 INFO:teuthology.orchestra.run.smithi070.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 569 kB/s | 80 kB 00:00 2024-09-13T15:41:27.946 INFO:teuthology.orchestra.run.smithi070.stdout:(2/2): ceph-fuse-17.2.7-1627.gbbdadfa3.el9.x86_ 2.7 MB/s | 813 kB 00:00 2024-09-13T15:41:27.947 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-13T15:41:27.947 INFO:teuthology.orchestra.run.smithi070.stdout:Total 2.0 MB/s | 893 kB 00:00 2024-09-13T15:41:27.959 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-13T15:41:27.981 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-13T15:41:27.981 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-13T15:41:28.068 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-13T15:41:28.069 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-13T15:41:28.252 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-13T15:41:28.393 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-13T15:41:28.460 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2/2 2024-09-13T15:41:29.137 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2/2 2024-09-13T15:41:29.137 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/2 2024-09-13T15:41:29.421 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-13T15:41:29.421 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:29.422 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-13T15:41:29.422 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-13T15:41:29.422 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:29.422 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:29.675 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librados-devel 2024-09-13T15:41:30.240 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:57 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:30.370 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:30.371 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:30.371 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-13T15:41:30.371 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:30.371 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-13T15:41:30.371 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 126 k 2024-09-13T15:41:30.371 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:30.371 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-13T15:41:30.371 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:30.371 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-13T15:41:30.371 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:30.372 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 126 k 2024-09-13T15:41:30.372 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 448 k 2024-09-13T15:41:30.372 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-13T15:41:30.575 INFO:teuthology.orchestra.run.smithi070.stdout:librados-devel-17.2.7-1627.gbbdadfa3.el9.x86_64 619 kB/s | 126 kB 00:00 2024-09-13T15:41:30.576 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-13T15:41:30.576 INFO:teuthology.orchestra.run.smithi070.stdout:Total 615 kB/s | 126 kB 00:00 2024-09-13T15:41:30.576 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-13T15:41:30.581 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-13T15:41:30.582 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-13T15:41:30.638 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-13T15:41:30.638 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-13T15:41:30.778 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-13T15:41:30.873 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/1 2024-09-13T15:41:31.268 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/1 2024-09-13T15:41:31.483 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/1 2024-09-13T15:41:31.483 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:31.483 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-13T15:41:31.483 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:41:31.483 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:31.483 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:31.703 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install libcephfs2 2024-09-13T15:41:32.268 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:59 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:32.361 INFO:teuthology.orchestra.run.smithi070.stdout:Package libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 is already installed. 2024-09-13T15:41:32.403 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:32.404 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-13T15:41:32.405 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:32.479 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install libcephfs-devel 2024-09-13T15:41:33.032 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:00 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:33.164 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:33.165 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:33.165 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-13T15:41:33.165 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:33.165 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-13T15:41:33.165 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 25 k 2024-09-13T15:41:33.165 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:33.165 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-13T15:41:33.166 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:33.166 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-13T15:41:33.166 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:33.166 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 25 k 2024-09-13T15:41:33.166 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 110 k 2024-09-13T15:41:33.166 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-13T15:41:33.314 INFO:teuthology.orchestra.run.smithi070.stdout:libcephfs-devel-17.2.7-1627.gbbdadfa3.el9.x86_6 168 kB/s | 25 kB 00:00 2024-09-13T15:41:33.315 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-13T15:41:33.315 INFO:teuthology.orchestra.run.smithi070.stdout:Total 167 kB/s | 25 kB 00:00 2024-09-13T15:41:33.315 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-13T15:41:33.318 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-13T15:41:33.319 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-13T15:41:33.320 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-13T15:41:33.320 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-13T15:41:33.406 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-13T15:41:33.509 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/1 2024-09-13T15:41:33.844 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/1 2024-09-13T15:41:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/1 2024-09-13T15:41:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:34.056 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-13T15:41:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:41:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:34.056 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:34.219 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librados2 2024-09-13T15:41:34.778 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:01 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:34.868 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 is already installed. 2024-09-13T15:41:34.909 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:34.910 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-13T15:41:34.910 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:34.973 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librbd1 2024-09-13T15:41:35.525 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:02 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:35.614 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 is already installed. 2024-09-13T15:41:35.655 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:35.656 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-13T15:41:35.656 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:35.719 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rados 2024-09-13T15:41:36.276 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:03 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:36.367 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rados-2:17.2.7-1627.gbbdadfa3.el9.x86_64 is already installed. 2024-09-13T15:41:36.408 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:36.409 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-13T15:41:36.409 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:36.470 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rgw 2024-09-13T15:41:37.027 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:04 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:37.119 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 is already installed. 2024-09-13T15:41:37.161 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:37.162 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-13T15:41:37.162 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:37.231 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-cephfs 2024-09-13T15:41:37.787 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:04 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:37.875 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-cephfs-2:17.2.7-1627.gbbdadfa3.el9.x86_64 is already installed. 2024-09-13T15:41:37.917 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:37.918 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-13T15:41:37.918 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:37.977 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rbd 2024-09-13T15:41:38.529 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:05 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:38.620 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 is already installed. 2024-09-13T15:41:38.661 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:38.662 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-13T15:41:38.662 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:38.718 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-fuse 2024-09-13T15:41:39.273 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:06 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:39.403 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:39.404 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:39.404 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-09-13T15:41:39.404 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:39.404 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-13T15:41:39.404 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 82 k 2024-09-13T15:41:39.404 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:39.404 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-13T15:41:39.404 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:39.404 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-13T15:41:39.404 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:39.404 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 82 k 2024-09-13T15:41:39.404 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 222 k 2024-09-13T15:41:39.405 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-13T15:41:39.608 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-fuse-17.2.7-1627.gbbdadfa3.el9.x86_64.rpm 401 kB/s | 82 kB 00:00 2024-09-13T15:41:39.609 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-13T15:41:39.609 INFO:teuthology.orchestra.run.smithi070.stdout:Total 398 kB/s | 82 kB 00:00 2024-09-13T15:41:39.609 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-13T15:41:39.617 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-13T15:41:39.617 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-13T15:41:39.673 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-13T15:41:39.673 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-13T15:41:39.812 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-13T15:41:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/1 2024-09-13T15:41:40.286 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/1 2024-09-13T15:41:40.530 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/1 2024-09-13T15:41:40.530 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:40.530 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-13T15:41:40.531 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:41:40.531 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:40.531 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:40.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-mirror 2024-09-13T15:41:41.323 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:08 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:41.459 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:41.460 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:41.460 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-13T15:41:41.460 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:41.460 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-13T15:41:41.460 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 2.9 M 2024-09-13T15:41:41.460 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:41.460 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-13T15:41:41.460 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:41.460 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-13T15:41:41.460 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:41.460 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 2.9 M 2024-09-13T15:41:41.460 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 12 M 2024-09-13T15:41:41.461 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-13T15:41:41.787 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-mirror-17.2.7-1627.gbbdadfa3.el9.x86_64.rpm 9.0 MB/s | 2.9 MB 00:00 2024-09-13T15:41:41.787 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-13T15:41:41.788 INFO:teuthology.orchestra.run.smithi070.stdout:Total 9.0 MB/s | 2.9 MB 00:00 2024-09-13T15:41:41.788 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-13T15:41:41.797 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-13T15:41:41.797 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-13T15:41:41.872 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-13T15:41:41.903 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-13T15:41:42.281 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-13T15:41:42.454 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/1 2024-09-13T15:41:42.479 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/1 2024-09-13T15:41:42.479 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T15:41:42.479 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T15:41:42.479 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T15:41:42.480 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T15:41:42.480 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:43.462 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/1 2024-09-13T15:41:43.462 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:43.463 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-13T15:41:43.463 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:41:43.463 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:43.463 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:43.667 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-nbd 2024-09-13T15:41:44.241 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:11 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:44.372 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:44.373 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:44.373 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-09-13T15:41:44.373 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-13T15:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:17.2.7-1627.gbbdadfa3.el9 ceph 165 k 2024-09-13T15:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-13T15:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-13T15:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 165 k 2024-09-13T15:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 477 k 2024-09-13T15:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-13T15:41:44.607 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-nbd-17.2.7-1627.gbbdadfa3.el9.x86_64.rpm 710 kB/s | 165 kB 00:00 2024-09-13T15:41:44.607 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-13T15:41:44.608 INFO:teuthology.orchestra.run.smithi070.stdout:Total 705 kB/s | 165 kB 00:00 2024-09-13T15:41:44.608 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-13T15:41:44.615 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-13T15:41:44.615 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-13T15:41:44.673 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-13T15:41:44.673 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-13T15:41:44.834 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-13T15:41:44.937 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/1 2024-09-13T15:41:45.364 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/1 2024-09-13T15:41:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 1/1 2024-09-13T15:41:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:45.613 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-13T15:41:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 2024-09-13T15:41:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:45.614 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:45.877 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librados2 2024-09-13T15:41:46.454 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:13 ago on Fri 13 Sep 2024 03:40:33 PM UTC. 2024-09-13T15:41:46.547 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 is already installed. 2024-09-13T15:41:46.591 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:41:46.592 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-13T15:41:46.592 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:41:46.653 DEBUG:teuthology.parallel:result is None 2024-09-13T15:41:46.654 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T15:41:46.654 INFO:teuthology.packaging:ref: None 2024-09-13T15:41:46.654 INFO:teuthology.packaging:tag: None 2024-09-13T15:41:46.654 INFO:teuthology.packaging:branch: quincy 2024-09-13T15:41:46.654 INFO:teuthology.packaging:sha1: 50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T15:41:46.654 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-09-13T15:41:46.867 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T15:41:46.888 INFO:teuthology.orchestra.run.smithi070.stdout:17.2.7-1627.gbbdadfa3.el9 2024-09-13T15:41:46.889 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1627.gbbdadfa3.el9 2024-09-13T15:41:46.889 INFO:teuthology.task.install:The correct ceph version 17.2.7-1627.gbbdadfa3 is installed. 2024-09-13T15:41:46.891 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T15:41:46.891 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:41:46.891 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T15:41:46.954 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T15:41:46.954 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:41:46.954 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T15:41:47.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T15:41:47.102 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T15:41:47.102 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:41:47.103 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T15:41:47.168 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T15:41:47.238 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T15:41:47.239 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:41:47.239 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T15:41:47.308 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T15:41:47.336 INFO:teuthology.run_tasks:Running task print... 2024-09-13T15:41:47.949 INFO:teuthology.task.print:**** done installing quincy 2024-09-13T15:41:47.950 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T15:41:48.120 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T15:41:48.121 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /var/log/ceph 2024-09-13T15:41:48.151 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T15:41:48.151 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T15:41:48.220 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T15:41:48.220 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T15:41:48.297 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T15:41:48.297 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T15:41:48.297 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903753', 'branch': 'wip-rishabh-testing-20240910.095512-reef', 'description': 'fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/quincy 1-upgrade}}', 'email': None, 'first_in_suite': False, 'job_id': '7903753', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'install ceph/quincy latest\n'}], 'name': 'rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240910.095512-reef'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240910.095512-reef', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3']], 'seed': 1152, 'sha1': '50bbe7f395997dcd871813949c49addbbf630361', 'sleep_before_teardown': 0, 'subset': '984/1024', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240910.095512-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_50bbe7f395997dcd871813949c49addbbf630361/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '50bbe7f395997dcd871813949c49addbbf630361', 'targets': {'smithi070.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGYfV3u+lofsJ2Y5eqDVRbxn2FHE5sUjgnZA8eB2AzM+gitm8GrJxaSI77j03KVieidlNDdKexVafxYub7tmiug='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'branch': 'quincy', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-volume'], 'extra_packages': ['librados2'], 'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'}}, {'print': '**** done installing quincy'}, {'ceph': {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'log-ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'flavor': 'default', 'fs': 'xfs', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361', 'cluster': 'ceph'}}, {'exec': {'osd.0': ['ceph osd set-require-min-compat-client quincy']}}, {'print': '**** done ceph'}, {'print': '*** upgrading, no cephfs present'}, {'exec': {'mon.a': ['ceph fs dump']}}, {'install.upgrade': {'mon.a': None}}, {'print': '**** done install.upgrade'}, {'ceph.restart': {'daemons': ['mon.*', 'mgr.*'], 'mon-health-to-clog': False, 'wait-for-healthy': False}}, {'ceph.healthy': None}, {'ceph.restart': {'daemons': ['osd.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'exec': {'mon.a': ['ceph versions', 'ceph osd dump -f json-pretty', 'ceph fs dump', 'ceph osd require-osd-release reef', 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done']}}, {'ceph.healthy': None}, {'print': '**** done ceph.restart'}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-13_14:36:41', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T15:41:48.298 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T15:41:48.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T15:41:48.415 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:41:48.415 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T15:41:48.473 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T15:41:48.473 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2024-09-13T15:41:48.529 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T15:41:48.530 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T15:41:48.530 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-13T15:41:48.530 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T15:41:48.530 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T15:41:48.530 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-13 15:41:42.732130935 +0000 2024-09-13T15:41:48.530 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-13 15:39:56.410639711 +0000 2024-09-13T15:41:48.530 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-13 15:39:56.410639711 +0000 2024-09-13T15:41:48.530 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-13 15:39:56.410639711 +0000 2024-09-13T15:41:48.530 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T15:41:48.595 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-13T15:41:48.595 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-13T15:41:48.595 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 8.9528e-05 s, 5.7 MB/s 2024-09-13T15:41:48.596 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T15:41:48.654 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2024-09-13T15:41:48.711 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T15:41:48.711 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T15:41:48.711 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-13T15:41:48.711 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T15:41:48.711 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T15:41:48.711 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-13 15:41:42.732130935 +0000 2024-09-13T15:41:48.711 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-13 15:39:56.726629857 +0000 2024-09-13T15:41:48.711 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-13 15:39:56.726629857 +0000 2024-09-13T15:41:48.711 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-13 15:39:56.726629857 +0000 2024-09-13T15:41:48.712 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T15:41:48.776 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-13T15:41:48.776 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-13T15:41:48.776 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000103984 s, 4.9 MB/s 2024-09-13T15:41:48.777 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T15:41:48.833 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2024-09-13T15:41:48.889 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T15:41:48.890 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T15:41:48.890 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-13T15:41:48.890 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T15:41:48.890 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T15:41:48.890 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-13 15:41:42.733130904 +0000 2024-09-13T15:41:48.890 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-13 15:39:57.044619941 +0000 2024-09-13T15:41:48.890 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-13 15:39:57.044619941 +0000 2024-09-13T15:41:48.890 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-13 15:39:57.044619941 +0000 2024-09-13T15:41:48.890 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T15:41:48.953 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-13T15:41:48.953 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-13T15:41:48.953 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000104251 s, 4.9 MB/s 2024-09-13T15:41:48.954 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T15:41:49.011 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2024-09-13T15:41:49.066 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T15:41:49.066 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T15:41:49.066 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-13T15:41:49.066 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T15:41:49.067 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T15:41:49.067 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-13 15:41:42.733130904 +0000 2024-09-13T15:41:49.067 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-13 15:39:57.365609931 +0000 2024-09-13T15:41:49.067 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-13 15:39:57.365609931 +0000 2024-09-13T15:41:49.067 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-13 15:39:57.365609931 +0000 2024-09-13T15:41:49.067 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T15:41:49.130 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-13T15:41:49.130 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-13T15:41:49.130 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000108155 s, 4.7 MB/s 2024-09-13T15:41:49.131 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T15:41:49.188 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T15:41:49.188 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-09-13T15:41:49.188 INFO:tasks.ceph:Generating config... 2024-09-13T15:41:49.189 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-13T15:41:49.189 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-09-13T15:41:49.189 INFO:tasks.ceph:[global] bluestore warn on legacy statfs = False 2024-09-13T15:41:49.190 INFO:tasks.ceph:[global] bluestore warn on no per pool omap = False 2024-09-13T15:41:49.190 INFO:tasks.ceph:[global] mon pg warn min per osd = 0 2024-09-13T15:41:49.190 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-13T15:41:49.190 INFO:tasks.ceph:[client] debug client = 20 2024-09-13T15:41:49.190 INFO:tasks.ceph:[client] debug ms = 1 2024-09-13T15:41:49.190 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-13T15:41:49.190 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-13T15:41:49.190 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-13T15:41:49.190 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-13T15:41:49.190 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-13T15:41:49.190 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-13T15:41:49.190 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-13T15:41:49.190 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-13T15:41:49.191 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-13T15:41:49.191 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-13T15:41:49.191 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-13T15:41:49.191 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-13T15:41:49.191 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-13T15:41:49.191 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T15:41:49.191 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T15:41:49.191 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T15:41:49.191 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T15:41:49.192 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T15:41:49.192 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-13T15:41:49.192 INFO:tasks.ceph:[mon] mon warn on osd down out interval zero = False 2024-09-13T15:41:49.192 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-13T15:41:49.192 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-13T15:41:49.192 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-13T15:41:49.192 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-13T15:41:49.192 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T15:41:49.192 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T15:41:49.192 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T15:41:49.193 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T15:41:49.193 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T15:41:49.193 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T15:41:49.193 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T15:41:49.193 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T15:41:49.193 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T15:41:49.193 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T15:41:49.193 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T15:41:49.193 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T15:41:49.193 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-13T15:41:49.193 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T15:41:49.194 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T15:41:49.274 INFO:teuthology.orchestra.run.smithi070.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T15:41:49.277 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T15:41:49.315 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T15:41:49.379 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.70'), ('mon.b', '[v2:172.21.15.70:3301,v1:172.21.15.70:6790]'), ('mon.c', '[v2:172.21.15.70:3302,v1:172.21.15.70:6791]')] 2024-09-13T15:41:49.380 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790],[v2:172.21.15.70:3302,v1:172.21.15.70:6791]', 'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-13T15:41:49.381 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:41:49.381 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T15:41:49.436 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.70 --addv b '[v2:172.21.15.70:3301,v1:172.21.15.70:6790]' --addv c '[v2:172.21.15.70:3302,v1:172.21.15.70:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T15:41:49.517 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T15:41:49.517 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool: generated fsid 45bbcdad-f4fb-4639-ab42-ccda8de151dd 2024-09-13T15:41:49.517 INFO:teuthology.orchestra.run.smithi070.stdout:setting min_mon_release = octopus 2024-09-13T15:41:49.518 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 0 2024-09-13T15:41:49.518 INFO:teuthology.orchestra.run.smithi070.stdout:fsid 45bbcdad-f4fb-4639-ab42-ccda8de151dd 2024-09-13T15:41:49.518 INFO:teuthology.orchestra.run.smithi070.stdout:last_changed 2024-09-13T15:41:49.517210+0000 2024-09-13T15:41:49.518 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-09-13T15:41:49.517210+0000 2024-09-13T15:41:49.518 INFO:teuthology.orchestra.run.smithi070.stdout:min_mon_release 15 (octopus) 2024-09-13T15:41:49.518 INFO:teuthology.orchestra.run.smithi070.stdout:election_strategy: 1 2024-09-13T15:41:49.518 INFO:teuthology.orchestra.run.smithi070.stdout:0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.a 2024-09-13T15:41:49.518 INFO:teuthology.orchestra.run.smithi070.stdout:1: [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] mon.b 2024-09-13T15:41:49.518 INFO:teuthology.orchestra.run.smithi070.stdout:2: [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] mon.c 2024-09-13T15:41:49.518 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T15:41:49.519 DEBUG:teuthology.orchestra.run.smithi070:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T15:41:49.575 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 45bbcdad-f4fb-4639-ab42-ccda8de151dd... 2024-09-13T15:41:49.576 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T15:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2024-09-13T15:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: chdir = "" 2024-09-13T15:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T15:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: auth supported = cephx 2024-09-13T15:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: filestore xattr use omap = true 2024-09-13T15:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: mon clock drift allowed = 1.000 2024-09-13T15:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: osd crush chooseleaf type = 0 2024-09-13T15:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: auth debug = true 2024-09-13T15:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on old message = true 2024-09-13T15:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on bug = true 2024-09-13T15:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T15:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: mon pg warn max object skew = 0 2024-09-13T15:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T15:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T15:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default size = 2 2024-09-13T15:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow primary affinity = true 2024-09-13T15:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow pg remap = true 2024-09-13T15:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on legacy crush tunables = false 2024-09-13T15:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on crush straw calc version zero = false 2024-09-13T15:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on no sortbitwise = false 2024-09-13T15:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on osd down out interval zero = false 2024-09-13T15:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on too few osds = false 2024-09-13T15:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T15:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T15:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: mon_allow_pool_size_one = true 2024-09-13T15:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T15:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: osd default data pool replay window = 5 2024-09-13T15:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: mon allow pool delete = true 2024-09-13T15:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: mon cluster log file level = debug 2024-09-13T15:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: debug asserts on shutdown = true 2024-09-13T15:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: mon health detail to clog = false 2024-09-13T15:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: mon host = "172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790],[v2:172.21.15.70:3302,v1:172.21.15.70:6791]" 2024-09-13T15:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on pool no app = False 2024-09-13T15:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: ms bind msgr2 = False 2024-09-13T15:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore warn on legacy statfs = False 2024-09-13T15:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore warn on no per pool omap = False 2024-09-13T15:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: mon pg warn min per osd = 0 2024-09-13T15:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = 45bbcdad-f4fb-4639-ab42-ccda8de151dd 2024-09-13T15:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout:[osd] 2024-09-13T15:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: osd journal size = 100 2024-09-13T15:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub load threshold = 5.0 2024-09-13T15:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub max interval = 600 2024-09-13T15:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: osd mclock profile = high_recovery_ops 2024-09-13T15:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: osd recover clone overlap = true 2024-09-13T15:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: osd recovery max chunk = 1048576 2024-09-13T15:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug shutdown = true 2024-09-13T15:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug op order = true 2024-09-13T15:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug verify stray on activate = true 2024-09-13T15:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: osd open classes on start = true 2024-09-13T15:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug pg log writeout = true 2024-09-13T15:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: osd deep scrub update digest min age = 30 2024-09-13T15:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: osd map max advance = 10 2024-09-13T15:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: journal zero on create = true 2024-09-13T15:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: filestore ondisk finisher threads = 3 2024-09-13T15:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: filestore apply finisher threads = 3 2024-09-13T15:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: bdev debug aio = true 2024-09-13T15:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug misdirected ops = true 2024-09-13T15:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: bdev async discard = True 2024-09-13T15:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: bdev enable discard = True 2024-09-13T15:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore allocator = bitmap 2024-09-13T15:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore block size = 96636764160 2024-09-13T15:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore fsck on mount = True 2024-09-13T15:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluefs = 1/20 2024-09-13T15:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluestore = 1/20 2024-09-13T15:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-13T15:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: debug osd = 20 2024-09-13T15:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: debug rocksdb = 4/10 2024-09-13T15:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T15:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd full ratio = 0.9 2024-09-13T15:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd nearfull ratio = 0.8 2024-09-13T15:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: osd failsafe full ratio = 0.95 2024-09-13T15:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: osd objectstore = bluestore 2024-09-13T15:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2024-09-13T15:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr] 2024-09-13T15:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-13T15:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: debug mgr = 20 2024-09-13T15:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2024-09-13T15:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2024-09-13T15:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2024-09-13T15:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2024-09-13T15:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: mgr/telemetry/nag = false 2024-09-13T15:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2024-09-13T15:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout:[mon] 2024-09-13T15:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-13T15:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2024-09-13T15:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: debug paxos = 20 2024-09-13T15:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2024-09-13T15:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: mon data avail warn = 5 2024-09-13T15:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: mon mgr mkfs grace = 240 2024-09-13T15:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2024-09-13T15:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd reporter subtree level = osd 2024-09-13T15:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd prime pg temp = true 2024-09-13T15:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2024-09-13T15:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T15:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T15:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T15:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T15:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T15:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T15:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: # 1m isn't quite enough 2024-09-13T15:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: mon_down_mkfs_grace = 2m 2024-09-13T15:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_filestore_osds = false 2024-09-13T15:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: mon op complaint time = 120 2024-09-13T15:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on osd down out interval zero = False 2024-09-13T15:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on pool no app = False 2024-09-13T15:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout:[client] 2024-09-13T15:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: rgw cache enabled = true 2024-09-13T15:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable ops log = true 2024-09-13T15:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable usage log = true 2024-09-13T15:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T15:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T15:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: client mount timeout = 600 2024-09-13T15:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2024-09-13T15:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-13T15:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 900 2024-09-13T15:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 900 2024-09-13T15:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.a] 2024-09-13T15:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.b] 2024-09-13T15:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.c] 2024-09-13T15:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout:[mds] 2024-09-13T15:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: debug mds = 20 2024-09-13T15:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: debug mds balancer = 20 2024-09-13T15:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-09-13T15:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug frag = True 2024-09-13T15:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug scatterstat = True 2024-09-13T15:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: mds op complaint time = 180 2024-09-13T15:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: mds verify scatter = True 2024-09-13T15:41:49.671 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2024-09-13T15:41:49.671 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 900 2024-09-13T15:41:49.671 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 900 2024-09-13T15:41:49.672 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T15:41:49.672 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T15:41:49.762 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T15:41:49.762 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:41:49.762 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T15:41:49.777 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:41:49.777 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T15:41:49.832 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T15:41:49.832 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:41:49.832 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T15:41:49.832 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T15:41:49.906 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:41:49.906 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T15:41:49.962 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T15:41:49.962 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T15:41:49.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T15:41:50.049 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T15:41:50.051 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T15:41:50.097 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T15:41:50.100 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T15:41:50.100 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T15:41:50.100 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T15:41:50.100 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-09-13T15:41:50.100 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T15:41:50.125 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T15:41:50.125 INFO:tasks.ceph:role: osd.0 2024-09-13T15:41:50.125 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T15:41:50.125 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T15:41:50.189 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T15:41:50.189 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T15:41:50.189 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T15:41:50.189 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T15:41:50.189 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T15:41:50.189 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-13T15:41:50.190 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T15:41:50.190 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T15:41:50.190 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T15:41:50.190 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T15:41:50.221 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-13T15:41:50.223 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-13T15:41:50.223 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T15:41:50.292 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T15:41:50.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T15:41:50.427 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T15:41:50.427 INFO:tasks.ceph:role: osd.1 2024-09-13T15:41:50.427 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T15:41:50.427 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T15:41:50.492 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T15:41:50.492 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T15:41:50.492 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T15:41:50.493 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T15:41:50.493 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T15:41:50.493 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-13T15:41:50.493 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T15:41:50.493 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T15:41:50.493 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T15:41:50.493 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T15:41:50.524 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-13T15:41:50.526 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-13T15:41:50.526 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T15:41:50.594 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T15:41:50.660 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T15:41:50.726 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T15:41:50.726 INFO:tasks.ceph:role: osd.2 2024-09-13T15:41:50.726 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T15:41:50.726 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T15:41:50.789 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T15:41:50.789 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T15:41:50.789 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T15:41:50.789 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T15:41:50.790 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T15:41:50.790 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-13T15:41:50.790 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T15:41:50.790 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T15:41:50.790 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T15:41:50.790 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T15:41:50.822 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-13T15:41:50.824 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-13T15:41:50.824 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T15:41:50.892 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T15:41:50.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T15:41:51.027 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T15:41:51.027 INFO:tasks.ceph:role: osd.3 2024-09-13T15:41:51.027 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T15:41:51.027 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T15:41:51.093 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T15:41:51.093 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T15:41:51.093 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T15:41:51.093 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T15:41:51.093 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T15:41:51.093 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-09-13T15:41:51.093 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T15:41:51.093 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T15:41:51.093 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T15:41:51.093 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T15:41:51.125 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-09-13T15:41:51.126 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-09-13T15:41:51.127 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T15:41:51.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T15:41:51.264 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T15:41:51.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:51.370+0000 7f0661835440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T15:41:51.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:51.370+0000 7f0661835440 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T15:41:51.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:51.370+0000 7f0661835440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T15:41:51.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:51.370+0000 7f0661835440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T15:41:51.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:51.370+0000 7f0661835440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T15:41:51.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:51.370+0000 7f0661835440 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T15:41:51.894 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:51.892+0000 7f0661835440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T15:41:52.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:52.654+0000 7f0661835440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T15:41:53.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:53.420+0000 7f0661835440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T15:41:54.219 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T15:41:54.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T15:41:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:54.330+0000 7f99e5a0e440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T15:41:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:54.330+0000 7f99e5a0e440 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T15:41:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:54.330+0000 7f99e5a0e440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T15:41:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:54.330+0000 7f99e5a0e440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T15:41:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:54.330+0000 7f99e5a0e440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T15:41:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:54.331+0000 7f99e5a0e440 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T15:41:54.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:54.856+0000 7f99e5a0e440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T15:41:55.618 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:55.617+0000 7f99e5a0e440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T15:41:56.381 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:56.380+0000 7f99e5a0e440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T15:41:57.175 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T15:41:57.245 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T15:41:57.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:57.331+0000 7f5ede61e440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T15:41:57.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:57.331+0000 7f5ede61e440 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T15:41:57.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:57.331+0000 7f5ede61e440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T15:41:57.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:57.331+0000 7f5ede61e440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T15:41:57.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:57.331+0000 7f5ede61e440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T15:41:57.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:57.332+0000 7f5ede61e440 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T15:41:57.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:57.855+0000 7f5ede61e440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T15:41:58.620 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:58.619+0000 7f5ede61e440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T15:41:59.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:41:59.381+0000 7f5ede61e440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T15:42:00.179 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T15:42:00.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T15:42:00.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:00.292+0000 7f582ac37440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T15:42:00.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:00.292+0000 7f582ac37440 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T15:42:00.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:00.292+0000 7f582ac37440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T15:42:00.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:00.292+0000 7f582ac37440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T15:42:00.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:00.292+0000 7f582ac37440 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T15:42:00.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:00.292+0000 7f582ac37440 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T15:42:00.816 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:00.815+0000 7f582ac37440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T15:42:01.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:01.577+0000 7f582ac37440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T15:42:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:02.340+0000 7f582ac37440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-13T15:42:03.134 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T15:42:03.167 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T15:42:03.167 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:42:03.167 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T15:42:03.235 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:42:03.236 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T15:42:03.304 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:42:03.304 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T15:42:03.371 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:42:03.371 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T15:42:03.438 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:42:03.438 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T15:42:03.504 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:42:03.504 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T15:42:03.569 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T15:42:03.570 DEBUG:teuthology.orchestra.run.smithi070:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T15:42:03.635 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.x] 2024-09-13T15:42:03.635 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQA+XeRmpS7rAhAA0tFPzj9GWp4MnzSY4LSo2Q== 2024-09-13T15:42:03.635 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.y] 2024-09-13T15:42:03.635 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQA+XeRmM5/SBRAAXn3cLVxIkcbefDV0wkPdCQ== 2024-09-13T15:42:03.635 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.0] 2024-09-13T15:42:03.636 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQA/XeRmoFYsFhAAKTKibyqNo2VuIs/IZiRoQQ== 2024-09-13T15:42:03.636 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.1] 2024-09-13T15:42:03.636 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQBCXeRmbuvAExAAdl3ZjY9R5ClL6UJB93g/FQ== 2024-09-13T15:42:03.636 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.2] 2024-09-13T15:42:03.636 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQBFXeRm2VPPExAAUuxsYOsRzV+BrVmVC+v0FQ== 2024-09-13T15:42:03.636 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.3] 2024-09-13T15:42:03.636 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQBIXeRmsH14ERAAVQSp5SnRUqAX/ETVj5Tz3w== 2024-09-13T15:42:03.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T15:42:03.725 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T15:42:03.771 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T15:42:03.817 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T15:42:03.864 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T15:42:03.912 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T15:42:03.959 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T15:42:03.960 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T15:42:03.985 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T15:42:04.078 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T15:42:04.103 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T15:42:04.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T15:42:04.259 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T15:42:04.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T15:42:04.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T15:42:04.439 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T15:42:04.464 DEBUG:teuthology.orchestra.run.smithi070:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T15:42:04.519 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T15:42:04.519 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T15:42:04.519 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T15:42:04.561 INFO:tasks.ceph.mon.a:Started 2024-09-13T15:42:04.562 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T15:42:04.562 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T15:42:04.564 INFO:tasks.ceph.mon.b:Started 2024-09-13T15:42:04.564 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T15:42:04.564 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T15:42:04.568 INFO:tasks.ceph.mon.c:Started 2024-09-13T15:42:04.568 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T15:42:04.569 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T15:42:04.569 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T15:42:04.571 INFO:tasks.ceph.mgr.x:Started 2024-09-13T15:42:04.571 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T15:42:04.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T15:42:04.573 INFO:tasks.ceph.mgr.y:Started 2024-09-13T15:42:04.574 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T15:42:04.574 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T15:42:04.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.689+0000 7f1e27f5d640 1 Processor -- start 2024-09-13T15:42:04.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.689+0000 7f1e27f5d640 1 -- start start 2024-09-13T15:42:04.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.689+0000 7f1e27f5d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e2012a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:04.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.689+0000 7f1e27f5d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 0x7f1e200969d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:04.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.689+0000 7f1e27f5d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e20099130 con 0x7f1e20096f10 2024-09-13T15:42:04.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.689+0000 7f1e27f5d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e20098930 con 0x7f1e20129f80 2024-09-13T15:42:04.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.689+0000 7f1e27f5d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2009a120 con 0x7f1e2012a890 2024-09-13T15:42:04.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.689+0000 7f1e25cf9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e2012a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:04.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.689+0000 7f1e25cf9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e2012a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53374/0 (socket says 172.21.15.70:53374) 2024-09-13T15:42:04.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.689+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 learned_addr learned my addr 172.21.15.70:0/1357383132 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:04.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.689+0000 7f1e254f8640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 0x7f1e200969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:04.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.690+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 msgr2=0x7f1e2012a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:42:04.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.690+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 msgr2=0x7f1e2012a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:42:04.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.690+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e2012a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:04.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.690+0000 7f1e254f8640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 msgr2=0x7f1e200969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T15:42:04.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.690+0000 7f1e254f8640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 msgr2=0x7f1e200969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:42:04.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.690+0000 7f1e254f8640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 0x7f1e200969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:04.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.690+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e2012a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:42:04.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.690+0000 7f1e254f8640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 0x7f1e200969d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:42:04.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.891+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e2012a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:04.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.891+0000 7f1e254f8640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 0x7f1e200969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:04.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.891+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 msgr2=0x7f1e2012a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:42:04.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.891+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 msgr2=0x7f1e2012a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:42:04.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.891+0000 7f1e254f8640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 msgr2=0x7f1e200969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T15:42:04.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.891+0000 7f1e254f8640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 msgr2=0x7f1e200969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:42:04.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.891+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e2012a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:04.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.891+0000 7f1e254f8640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 0x7f1e200969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:04.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.891+0000 7f1e254f8640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 0x7f1e200969d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T15:42:04.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:04.891+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e2012a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T15:42:05.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:05.292+0000 7f1e254f8640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 0x7f1e200969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:05.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:05.292+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e2012a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:05.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:05.292+0000 7f1e254f8640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 msgr2=0x7f1e200969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:42:05.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:05.292+0000 7f1e254f8640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 msgr2=0x7f1e200969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:42:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:05.292+0000 7f1e254f8640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 0x7f1e200969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:05.292+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 msgr2=0x7f1e2012a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T15:42:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:05.292+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 msgr2=0x7f1e2012a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:42:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:05.292+0000 7f1e254f8640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 0x7f1e200969d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T15:42:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:05.292+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e2012a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:05.292+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e2012a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T15:42:06.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:06.093+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e2012a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:06.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:06.093+0000 7f1e254f8640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 0x7f1e200969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:06.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:06.093+0000 7f1e254f8640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 msgr2=0x7f1e200969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:42:06.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:06.093+0000 7f1e254f8640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 msgr2=0x7f1e200969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:42:06.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:06.093+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 msgr2=0x7f1e2012a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T15:42:06.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:06.093+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 msgr2=0x7f1e2012a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:42:06.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:06.093+0000 7f1e254f8640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 0x7f1e200969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:06.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:06.093+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e2012a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:06.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:06.093+0000 7f1e254f8640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 0x7f1e200969d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T15:42:06.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:06.093+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e2012a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T15:42:07.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.690+0000 7f1e177fe640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 msgr2=0x7f1e200969d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T15:42:07.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.690+0000 7f1e177fe640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 0x7f1e200969d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:07.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.690+0000 7f1e177fe640 1 -- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 msgr2=0x7f1e2012a350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T15:42:07.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.690+0000 7f1e177fe640 1 --2- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e2012a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.690+0000 7f1e177fe640 1 -- 172.21.15.70:0/1357383132 >> v1:172.21.15.70:6789/0 conn(0x7f1e20096f10 legacy=0x7f1e200973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.690+0000 7f1e177fe640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e04005a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.690+0000 7f1e177fe640 1 -- 172.21.15.70:0/1357383132 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e20098930 con 0x7f1e04003610 2024-09-13T15:42:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.690+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e04005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.690+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 msgr2=0x7f1e04005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:42:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.690+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 msgr2=0x7f1e04005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:42:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.690+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e04005a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.690+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e04005a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:42:07.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.891+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e04005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:07.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.891+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 msgr2=0x7f1e04005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:42:07.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.891+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 msgr2=0x7f1e04005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:42:07.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.891+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e04005a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:07.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:07.891+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e04005a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T15:42:08.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:08.292+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e04005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:08.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:08.292+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 msgr2=0x7f1e04005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:42:08.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:08.292+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 msgr2=0x7f1e04005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:42:08.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:08.292+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e04005a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:08.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:08.292+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e04005a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T15:42:09.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:09.093+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e04005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:09.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:09.093+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 msgr2=0x7f1e04005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:42:09.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:09.093+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 msgr2=0x7f1e04005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:42:09.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:09.093+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e04005a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:09.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:09.093+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e04005a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T15:42:10.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.690+0000 7f1e177fe640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 msgr2=0x7f1e04005a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T15:42:10.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.690+0000 7f1e177fe640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e04005a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:10.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.690+0000 7f1e177fe640 1 --2- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e04008360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:10.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.690+0000 7f1e177fe640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e040088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:10.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.690+0000 7f1e177fe640 1 -- 172.21.15.70:0/1357383132 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e200995e0 con 0x7f1e20096f10 2024-09-13T15:42:10.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.690+0000 7f1e177fe640 1 -- 172.21.15.70:0/1357383132 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e20098930 con 0x7f1e20129f80 2024-09-13T15:42:10.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.690+0000 7f1e177fe640 1 -- 172.21.15.70:0/1357383132 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2009a120 con 0x7f1e04003610 2024-09-13T15:42:10.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.690+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> v2:172.21.15.70:3300/0 conn(0x7f1e20129f80 0x7f1e04008360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:10.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.690+0000 7f1e254f8640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e040088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:10.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.690+0000 7f1e24cf7640 1 -- 172.21.15.70:0/1357383132 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866303022 0 0) 0x7f1e200995e0 con 0x7f1e20096f10 2024-09-13T15:42:10.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.690+0000 7f1e24cf7640 1 -- 172.21.15.70:0/1357383132 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e0c0034e0 con 0x7f1e20096f10 2024-09-13T15:42:10.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.690+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 msgr2=0x7f1e040088a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:10.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.690+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e040088a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:10.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.690+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 >> v1:172.21.15.70:6789/0 conn(0x7f1e20096f10 legacy=0x7f1e0400adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:10.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.690+0000 7f1e25cf9640 1 -- 172.21.15.70:0/1357383132 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e04005c40 con 0x7f1e20129f80 2024-09-13T15:42:10.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.692+0000 7f1e25cf9640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e20129f80 0x7f1e04008360 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c006fd0 tx=0x7f1e1c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24699a9910a3a3e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:10.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.692+0000 7f1e24cf7640 1 -- 172.21.15.70:0/1357383132 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c02c800 con 0x7f1e20129f80 2024-09-13T15:42:10.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.692+0000 7f1e24cf7640 1 -- 172.21.15.70:0/1357383132 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1c00c040 con 0x7f1e20129f80 2024-09-13T15:42:10.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.692+0000 7f1e24cf7640 1 -- 172.21.15.70:0/1357383132 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c02cbb0 con 0x7f1e20129f80 2024-09-13T15:42:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.693+0000 7f1e27f5d640 1 -- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e20129f80 msgr2=0x7f1e04008360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.693+0000 7f1e27f5d640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e20129f80 0x7f1e04008360 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c006fd0 tx=0x7f1e1c008040 comp rx=0 tx=0).stop 2024-09-13T15:42:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.693+0000 7f1e27f5d640 1 -- 172.21.15.70:0/1357383132 shutdown_connections 2024-09-13T15:42:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.693+0000 7f1e27f5d640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e040088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.693+0000 7f1e27f5d640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e20129f80 0x7f1e04008360 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.693+0000 7f1e27f5d640 1 --2- 172.21.15.70:0/1357383132 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e2012a890 0x7f1e200969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.693+0000 7f1e27f5d640 1 -- 172.21.15.70:0/1357383132 >> 172.21.15.70:0/1357383132 conn(0x7f1e2009d730 msgr2=0x7f1e2009db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.693+0000 7f1e27f5d640 1 -- 172.21.15.70:0/1357383132 shutdown_connections 2024-09-13T15:42:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.693+0000 7f1e27f5d640 1 -- 172.21.15.70:0/1357383132 wait complete. 2024-09-13T15:42:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.694+0000 7f1e27f5d640 1 Processor -- start 2024-09-13T15:42:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.694+0000 7f1e27f5d640 1 -- start start 2024-09-13T15:42:10.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.694+0000 7f1e27f5d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e20138ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:10.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.694+0000 7f1e27f5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e20096f10 0x7f1e20139400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:10.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.694+0000 7f1e27f5d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e20129f80 0x7f1e20139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:10.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.694+0000 7f1e27f5d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e20120050 con 0x7f1e20096f10 2024-09-13T15:42:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.694+0000 7f1e27f5d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e20120400 con 0x7f1e04003610 2024-09-13T15:42:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.694+0000 7f1e27f5d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e20129690 con 0x7f1e20129f80 2024-09-13T15:42:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.695+0000 7f1e25cf9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e20138ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.695+0000 7f1e254f8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e20096f10 0x7f1e20139400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.695+0000 7f1e254f8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e20096f10 0x7f1e20139400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53510/0 (socket says 172.21.15.70:53510) 2024-09-13T15:42:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.695+0000 7f1e254f8640 1 -- 172.21.15.70:0/2761132852 learned_addr learned my addr 172.21.15.70:0/2761132852 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.695+0000 7f1e264fa640 1 --2- 172.21.15.70:0/2761132852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e20129f80 0x7f1e20139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.695+0000 7f1e264fa640 1 -- 172.21.15.70:0/2761132852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e20129f80 msgr2=0x7f1e20139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T15:42:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.695+0000 7f1e264fa640 1 -- 172.21.15.70:0/2761132852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e20129f80 msgr2=0x7f1e20139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T15:42:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.695+0000 7f1e264fa640 1 --2- 172.21.15.70:0/2761132852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e20129f80 0x7f1e20139940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.695+0000 7f1e264fa640 1 --2- 172.21.15.70:0/2761132852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e20129f80 0x7f1e20139940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:42:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.695+0000 7f1e254f8640 1 -- 172.21.15.70:0/2761132852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e20129f80 msgr2=0x7f1e20139940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:42:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.695+0000 7f1e254f8640 1 --2- 172.21.15.70:0/2761132852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e20129f80 0x7f1e20139940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:10.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.695+0000 7f1e254f8640 1 -- 172.21.15.70:0/2761132852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 msgr2=0x7f1e20138ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:10.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.695+0000 7f1e254f8640 1 --2- 172.21.15.70:0/2761132852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e20138ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:10.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.695+0000 7f1e254f8640 1 -- 172.21.15.70:0/2761132852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c00e040 con 0x7f1e20096f10 2024-09-13T15:42:10.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.695+0000 7f1e254f8640 1 --2- 172.21.15.70:0/2761132852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e20096f10 0x7f1e20139400 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1e10016a80 tx=0x7f1e10008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:10.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.696+0000 7f1e15ffb640 1 -- 172.21.15.70:0/2761132852 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000f6f0 con 0x7f1e20096f10 2024-09-13T15:42:10.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.696+0000 7f1e15ffb640 1 -- 172.21.15.70:0/2761132852 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1001b070 con 0x7f1e20096f10 2024-09-13T15:42:10.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.696+0000 7f1e27f5d640 1 -- 172.21.15.70:0/2761132852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e20120880 con 0x7f1e20096f10 2024-09-13T15:42:10.700 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.696+0000 7f1e15ffb640 1 -- 172.21.15.70:0/2761132852 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10014c10 con 0x7f1e20096f10 2024-09-13T15:42:10.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.696+0000 7f1e27f5d640 1 -- 172.21.15.70:0/2761132852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2011d270 con 0x7f1e20096f10 2024-09-13T15:42:10.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.696+0000 7f1e15ffb640 1 -- 172.21.15.70:0/2761132852 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f1e1001d030 con 0x7f1e20096f10 2024-09-13T15:42:10.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.696+0000 7f1e15ffb640 1 -- 172.21.15.70:0/2761132852 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1e10006520 con 0x7f1e20096f10 2024-09-13T15:42:10.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.697+0000 7f1e27f5d640 1 -- 172.21.15.70:0/2761132852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2005ed70 con 0x7f1e20096f10 2024-09-13T15:42:10.701 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.699+0000 7f1e15ffb640 1 -- 172.21.15.70:0/2761132852 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f1e1003e7a0 con 0x7f1e20096f10 2024-09-13T15:42:10.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.763+0000 7f1e27f5d640 1 -- 172.21.15.70:0/2761132852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1e201293d0 con 0x7f1e20096f10 2024-09-13T15:42:10.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.765+0000 7f1e15ffb640 1 -- 172.21.15.70:0/2761132852 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1e1002d020 con 0x7f1e20096f10 2024-09-13T15:42:10.766 INFO:teuthology.orchestra.run.smithi070.stderr:adjusted tunables profile to default 2024-09-13T15:42:10.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.766+0000 7f1e27f5d640 1 -- 172.21.15.70:0/2761132852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e20096f10 msgr2=0x7f1e20139400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:10.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.766+0000 7f1e27f5d640 1 --2- 172.21.15.70:0/2761132852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e20096f10 0x7f1e20139400 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1e10016a80 tx=0x7f1e10008040 comp rx=0 tx=0).stop 2024-09-13T15:42:10.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.766+0000 7f1e27f5d640 1 -- 172.21.15.70:0/2761132852 shutdown_connections 2024-09-13T15:42:10.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.766+0000 7f1e27f5d640 1 --2- 172.21.15.70:0/2761132852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e20129f80 0x7f1e20139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:10.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.766+0000 7f1e27f5d640 1 --2- 172.21.15.70:0/2761132852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e20096f10 0x7f1e20139400 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:10.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.766+0000 7f1e27f5d640 1 --2- 172.21.15.70:0/2761132852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e04003610 0x7f1e20138ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:10.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.766+0000 7f1e27f5d640 1 -- 172.21.15.70:0/2761132852 >> 172.21.15.70:0/2761132852 conn(0x7f1e2009d730 msgr2=0x7f1e20116290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:10.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.766+0000 7f1e27f5d640 1 -- 172.21.15.70:0/2761132852 shutdown_connections 2024-09-13T15:42:10.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:10.766+0000 7f1e27f5d640 1 -- 172.21.15.70:0/2761132852 wait complete. 2024-09-13T15:42:10.771 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:10.770+0000 7ffa221beec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T15:42:10.771 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:10.771+0000 7fb67c63bec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T15:42:10.780 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T15:42:10.780 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T15:42:10.780 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:42:10.780 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T15:42:10.805 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:42:10.805 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T15:42:10.814 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:10.813+0000 7fb67c63bec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T15:42:10.815 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:10.814+0000 7ffa221beec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T15:42:10.840 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:42:10.840 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T15:42:10.867 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:10.865+0000 7fb67c63bec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T15:42:10.869 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:10.868+0000 7ffa221beec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T15:42:10.894 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:42:10.894 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T15:42:10.963 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 265c6f1d-08a7-4c43-b2a6-2417ab814825 0 2024-09-13T15:42:10.995 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:10.994+0000 7fb67c63bec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T15:42:10.996 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:10.995+0000 7ffa221beec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T15:42:11.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.089+0000 7f26bed5d640 1 Processor -- start 2024-09-13T15:42:11.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.089+0000 7f26bed5d640 1 -- start start 2024-09-13T15:42:11.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.089+0000 7f26bed5d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f26b8068bf0 0x7f26b812f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.089+0000 7f26bed5d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b812f910 0x7f26b8131cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.089+0000 7f26bed5d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b8069de0 con 0x7f26b8068110 2024-09-13T15:42:11.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.089+0000 7f26bed5d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b80644a0 con 0x7f26b8068bf0 2024-09-13T15:42:11.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.089+0000 7f26bed5d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b808f070 con 0x7f26b812f910 2024-09-13T15:42:11.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.089+0000 7f26b7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f26b8068bf0 0x7f26b812f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.089+0000 7f26bd2fa640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b812f910 0x7f26b8131cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.089+0000 7f26b7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f26b8068bf0 0x7f26b812f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53520/0 (socket says 172.21.15.70:53520) 2024-09-13T15:42:11.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.089+0000 7f26bcaf9640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f26b8068110 0x7f26b80684e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40786/0 (socket says 172.21.15.70:40786) 2024-09-13T15:42:11.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.089+0000 7f26b7fff640 1 -- 172.21.15.70:0/1478647103 learned_addr learned my addr 172.21.15.70:0/1478647103 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:11.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.090+0000 7f26b7fff640 1 -- 172.21.15.70:0/1478647103 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b812f910 msgr2=0x7f26b8131cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:11.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.090+0000 7f26b7fff640 1 --2- 172.21.15.70:0/1478647103 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b812f910 0x7f26b8131cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.090+0000 7f26b7fff640 1 -- 172.21.15.70:0/1478647103 >> v1:172.21.15.70:6789/0 conn(0x7f26b8068110 legacy=0x7f26b80684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.090+0000 7f26b7fff640 1 -- 172.21.15.70:0/1478647103 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b806a710 con 0x7f26b8068bf0 2024-09-13T15:42:11.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.090+0000 7f26bd2fa640 1 --2- 172.21.15.70:0/1478647103 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b812f910 0x7f26b8131cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:11.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.090+0000 7f26b7fff640 1 --2- 172.21.15.70:0/1478647103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b8068bf0 0x7f26b812f3d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009910 tx=0x7f26a40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f61aefc6ed050b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:11.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.091+0000 7f26b77fe640 1 -- 172.21.15.70:0/1478647103 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a402ad20 con 0x7f26b8068bf0 2024-09-13T15:42:11.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.091+0000 7f26b77fe640 1 -- 172.21.15.70:0/1478647103 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a402ae80 con 0x7f26b8068bf0 2024-09-13T15:42:11.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.091+0000 7f26b77fe640 1 -- 172.21.15.70:0/1478647103 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4032670 con 0x7f26b8068bf0 2024-09-13T15:42:11.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.091+0000 7f26bed5d640 1 -- 172.21.15.70:0/1478647103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b8068bf0 msgr2=0x7f26b812f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.091+0000 7f26bed5d640 1 --2- 172.21.15.70:0/1478647103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b8068bf0 0x7f26b812f3d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009910 tx=0x7f26a40091d0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.091+0000 7f26bed5d640 1 -- 172.21.15.70:0/1478647103 shutdown_connections 2024-09-13T15:42:11.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.091+0000 7f26bed5d640 1 --2- 172.21.15.70:0/1478647103 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b812f910 0x7f26b8131cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.091+0000 7f26bed5d640 1 --2- 172.21.15.70:0/1478647103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b8068bf0 0x7f26b812f3d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.091+0000 7f26bed5d640 1 -- 172.21.15.70:0/1478647103 >> 172.21.15.70:0/1478647103 conn(0x7f26b8097d20 msgr2=0x7f26b8098120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:11.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.091+0000 7f26bed5d640 1 -- 172.21.15.70:0/1478647103 shutdown_connections 2024-09-13T15:42:11.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.091+0000 7f26bed5d640 1 -- 172.21.15.70:0/1478647103 wait complete. 2024-09-13T15:42:11.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.092+0000 7f26bed5d640 1 Processor -- start 2024-09-13T15:42:11.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.092+0000 7f26bed5d640 1 -- start start 2024-09-13T15:42:11.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bed5d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f26b8068110 0x7f26b812f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bed5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b8068bf0 0x7f26b8120300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bed5d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b812f910 0x7f26b8120840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bed5d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b81225d0 con 0x7f26b8068bf0 2024-09-13T15:42:11.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bed5d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b806ed20 con 0x7f26b812f910 2024-09-13T15:42:11.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bed5d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8122120 con 0x7f26b8068110 2024-09-13T15:42:11.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bcaf9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f26b8068110 0x7f26b812f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bd2fa640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b812f910 0x7f26b8120840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bcaf9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f26b8068110 0x7f26b812f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:34136/0 (socket says 172.21.15.70:34136) 2024-09-13T15:42:11.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bcaf9640 1 -- 172.21.15.70:0/828564105 learned_addr learned my addr 172.21.15.70:0/828564105 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:11.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26b7fff640 1 --2- 172.21.15.70:0/828564105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b8068bf0 0x7f26b8120300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bcaf9640 1 -- 172.21.15.70:0/828564105 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f26b8068110 msgr2=0x7f26b812f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T15:42:11.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bcaf9640 1 -- 172.21.15.70:0/828564105 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f26b8068110 msgr2=0x7f26b812f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T15:42:11.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bcaf9640 1 --2- 172.21.15.70:0/828564105 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f26b8068110 0x7f26b812f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:11.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bcaf9640 1 --2- 172.21.15.70:0/828564105 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f26b8068110 0x7f26b812f270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:42:11.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bd2fa640 1 -- 172.21.15.70:0/828564105 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f26b8068110 msgr2=0x7f26b812f270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:42:11.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bd2fa640 1 --2- 172.21.15.70:0/828564105 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f26b8068110 0x7f26b812f270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bd2fa640 1 -- 172.21.15.70:0/828564105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b8068bf0 msgr2=0x7f26b8120300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bd2fa640 1 --2- 172.21.15.70:0/828564105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b8068bf0 0x7f26b8120300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bd2fa640 1 -- 172.21.15.70:0/828564105 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b0009a20 con 0x7f26b812f910 2024-09-13T15:42:11.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.093+0000 7f26bd2fa640 1 --2- 172.21.15.70:0/828564105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b812f910 0x7f26b8120840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009f10 tx=0x7f26b000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:11.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.094+0000 7f26b4ff9640 1 -- 172.21.15.70:0/828564105 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b000cdb0 con 0x7f26b812f910 2024-09-13T15:42:11.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.094+0000 7f26b4ff9640 1 -- 172.21.15.70:0/828564105 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b000f040 con 0x7f26b812f910 2024-09-13T15:42:11.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.094+0000 7f26bed5d640 1 -- 172.21.15.70:0/828564105 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a4009450 con 0x7f26b812f910 2024-09-13T15:42:11.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.094+0000 7f26b4ff9640 1 -- 172.21.15.70:0/828564105 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b00146d0 con 0x7f26b812f910 2024-09-13T15:42:11.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.094+0000 7f26bed5d640 1 -- 172.21.15.70:0/828564105 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b806e4c0 con 0x7f26b812f910 2024-09-13T15:42:11.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.094+0000 7f26b4ff9640 1 -- 172.21.15.70:0/828564105 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f26b0003020 con 0x7f26b812f910 2024-09-13T15:42:11.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.094+0000 7f26b4ff9640 1 -- 172.21.15.70:0/828564105 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f26b001dc80 con 0x7f26b812f910 2024-09-13T15:42:11.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.095+0000 7f26bed5d640 1 -- 172.21.15.70:0/828564105 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b805ed70 con 0x7f26b812f910 2024-09-13T15:42:11.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.097+0000 7f26b4ff9640 1 -- 172.21.15.70:0/828564105 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f26b0018030 con 0x7f26b812f910 2024-09-13T15:42:11.109 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:11.107+0000 7fb67c63bec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T15:42:11.109 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:11.108+0000 7ffa221beec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T15:42:11.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.165+0000 7f26bed5d640 1 -- 172.21.15.70:0/828564105 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "265c6f1d-08a7-4c43-b2a6-2417ab814825", "id": 0} v 0) v1 -- 0x7f26b8070450 con 0x7f26b812f910 2024-09-13T15:42:11.169 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.168+0000 7f26b4ff9640 1 -- 172.21.15.70:0/828564105 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "265c6f1d-08a7-4c43-b2a6-2417ab814825", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f26b001d410 con 0x7f26b812f910 2024-09-13T15:42:11.169 INFO:teuthology.orchestra.run.smithi070.stdout:0 2024-09-13T15:42:11.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.169+0000 7f26bed5d640 1 -- 172.21.15.70:0/828564105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b812f910 msgr2=0x7f26b8120840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.169+0000 7f26bed5d640 1 --2- 172.21.15.70:0/828564105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b812f910 0x7f26b8120840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009f10 tx=0x7f26b000b040 comp rx=0 tx=0).stop 2024-09-13T15:42:11.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.169+0000 7f26bed5d640 1 -- 172.21.15.70:0/828564105 shutdown_connections 2024-09-13T15:42:11.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.169+0000 7f26bed5d640 1 --2- 172.21.15.70:0/828564105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f26b812f910 0x7f26b8120840 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.169+0000 7f26bed5d640 1 --2- 172.21.15.70:0/828564105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26b8068bf0 0x7f26b8120300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.169+0000 7f26bed5d640 1 --2- 172.21.15.70:0/828564105 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f26b8068110 0x7f26b812f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.169+0000 7f26bed5d640 1 -- 172.21.15.70:0/828564105 >> 172.21.15.70:0/828564105 conn(0x7f26b8097d20 msgr2=0x7f26b80912f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:11.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.169+0000 7f26bed5d640 1 -- 172.21.15.70:0/828564105 shutdown_connections 2024-09-13T15:42:11.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.169+0000 7f26bed5d640 1 -- 172.21.15.70:0/828564105 wait complete. 2024-09-13T15:42:11.180 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new ba4d90f9-6972-45df-800f-92ad808b06ec 1 2024-09-13T15:42:11.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.272+0000 7fa60435d640 1 Processor -- start 2024-09-13T15:42:11.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.272+0000 7fa60435d640 1 -- start start 2024-09-13T15:42:11.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.272+0000 7fa60435d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5fc094970 0x7fa5fc091560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.272+0000 7fa60435d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc091aa0 0x7fa5fc091ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa60435d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5fc1270d0 0x7fa5fc127520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa60435d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5fc092410 con 0x7fa5fc094970 2024-09-13T15:42:11.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa60435d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc06f2e0 con 0x7fa5fc1270d0 2024-09-13T15:42:11.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa60435d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc092f30 con 0x7fa5fc091aa0 2024-09-13T15:42:11.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6018f8640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc091aa0 0x7fa5fc091ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6020f9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5fc094970 0x7fa5fc091560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6020f9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5fc094970 0x7fa5fc091560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53536/0 (socket says 172.21.15.70:53536) 2024-09-13T15:42:11.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6028fa640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5fc1270d0 0x7fa5fc127520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6018f8640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc091aa0 0x7fa5fc091ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:34140/0 (socket says 172.21.15.70:34140) 2024-09-13T15:42:11.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6020f9640 1 -- 172.21.15.70:0/3162036183 learned_addr learned my addr 172.21.15.70:0/3162036183 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:11.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6018f8640 1 -- 172.21.15.70:0/3162036183 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc091aa0 msgr2=0x7fa5fc091ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T15:42:11.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6018f8640 1 -- 172.21.15.70:0/3162036183 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc091aa0 msgr2=0x7fa5fc091ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:42:11.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6018f8640 1 --2- 172.21.15.70:0/3162036183 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc091aa0 0x7fa5fc091ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:11.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6018f8640 1 --2- 172.21.15.70:0/3162036183 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc091aa0 0x7fa5fc091ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:42:11.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6028fa640 1 -- 172.21.15.70:0/3162036183 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc091aa0 msgr2=0x7fa5fc091ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T15:42:11.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6028fa640 1 --2- 172.21.15.70:0/3162036183 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc091aa0 0x7fa5fc091ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6028fa640 1 -- 172.21.15.70:0/3162036183 >> v2:172.21.15.70:3300/0 conn(0x7fa5fc094970 msgr2=0x7fa5fc091560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:11.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6028fa640 1 --2- 172.21.15.70:0/3162036183 >> v2:172.21.15.70:3300/0 conn(0x7fa5fc094970 0x7fa5fc091560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6028fa640 1 -- 172.21.15.70:0/3162036183 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc095230 con 0x7fa5fc1270d0 2024-09-13T15:42:11.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.273+0000 7fa6020f9640 1 --2- 172.21.15.70:0/3162036183 >> v2:172.21.15.70:3300/0 conn(0x7fa5fc094970 0x7fa5fc091560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:11.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.274+0000 7fa6028fa640 1 --2- 172.21.15.70:0/3162036183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5fc1270d0 0x7fa5fc127520 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009e60 tx=0x7fa5f4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f154bd1b1f47447c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:11.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.274+0000 7fa6010f7640 1 -- 172.21.15.70:0/3162036183 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4007550 con 0x7fa5fc1270d0 2024-09-13T15:42:11.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.274+0000 7fa6010f7640 1 -- 172.21.15.70:0/3162036183 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f402d050 con 0x7fa5fc1270d0 2024-09-13T15:42:11.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.274+0000 7fa6010f7640 1 -- 172.21.15.70:0/3162036183 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4007920 con 0x7fa5fc1270d0 2024-09-13T15:42:11.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.274+0000 7fa60435d640 1 -- 172.21.15.70:0/3162036183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5fc1270d0 msgr2=0x7fa5fc127520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.274+0000 7fa60435d640 1 --2- 172.21.15.70:0/3162036183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5fc1270d0 0x7fa5fc127520 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009e60 tx=0x7fa5f4009780 comp rx=0 tx=0).stop 2024-09-13T15:42:11.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.274+0000 7fa60435d640 1 -- 172.21.15.70:0/3162036183 shutdown_connections 2024-09-13T15:42:11.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.274+0000 7fa60435d640 1 --2- 172.21.15.70:0/3162036183 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5fc1270d0 0x7fa5fc127520 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.274+0000 7fa60435d640 1 --2- 172.21.15.70:0/3162036183 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc091aa0 0x7fa5fc091ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.274+0000 7fa60435d640 1 --2- 172.21.15.70:0/3162036183 >> v2:172.21.15.70:3300/0 conn(0x7fa5fc094970 0x7fa5fc091560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.274+0000 7fa60435d640 1 -- 172.21.15.70:0/3162036183 >> 172.21.15.70:0/3162036183 conn(0x7fa5fc095a60 msgr2=0x7fa5fc095e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:11.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.274+0000 7fa60435d640 1 -- 172.21.15.70:0/3162036183 shutdown_connections 2024-09-13T15:42:11.278 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.274+0000 7fa60435d640 1 -- 172.21.15.70:0/3162036183 wait complete. 2024-09-13T15:42:11.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.275+0000 7fa60435d640 1 Processor -- start 2024-09-13T15:42:11.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.275+0000 7fa60435d640 1 -- start start 2024-09-13T15:42:11.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.275+0000 7fa60435d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5fc091aa0 0x7fa5fc08e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa60435d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc094970 0x7fa5fc08ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa60435d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5fc1270d0 0x7fa5fc08f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa60435d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc06aca0 con 0x7fa5fc091aa0 2024-09-13T15:42:11.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa60435d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc06aa90 con 0x7fa5fc1270d0 2024-09-13T15:42:11.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa60435d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc06bf50 con 0x7fa5fc094970 2024-09-13T15:42:11.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa6020f9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5fc091aa0 0x7fa5fc08e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa6020f9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5fc091aa0 0x7fa5fc08e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53538/0 (socket says 172.21.15.70:53538) 2024-09-13T15:42:11.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa6020f9640 1 -- 172.21.15.70:0/1676764224 learned_addr learned my addr 172.21.15.70:0/1676764224 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:11.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa6018f8640 1 --2- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc094970 0x7fa5fc08ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa6028fa640 1 --2- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5fc1270d0 0x7fa5fc08f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa6018f8640 1 -- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc094970 msgr2=0x7fa5fc08ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T15:42:11.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa6018f8640 1 -- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc094970 msgr2=0x7fa5fc08ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T15:42:11.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa6018f8640 1 --2- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc094970 0x7fa5fc08ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:11.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa6018f8640 1 --2- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc094970 0x7fa5fc08ec10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:42:11.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa6020f9640 1 -- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc094970 msgr2=0x7fa5fc08ec10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:42:11.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa6020f9640 1 --2- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc094970 0x7fa5fc08ec10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa6020f9640 1 -- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5fc1270d0 msgr2=0x7fa5fc08f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa6020f9640 1 --2- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5fc1270d0 0x7fa5fc08f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa6020f9640 1 -- 172.21.15.70:0/1676764224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f0009450 con 0x7fa5fc091aa0 2024-09-13T15:42:11.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa6020f9640 1 --2- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5fc091aa0 0x7fa5fc08e6d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000ec80 tx=0x7fa5f00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:11.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa5ee7fc640 1 -- 172.21.15.70:0/1676764224 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0009730 con 0x7fa5fc091aa0 2024-09-13T15:42:11.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa5ee7fc640 1 -- 172.21.15.70:0/1676764224 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f000eec0 con 0x7fa5fc091aa0 2024-09-13T15:42:11.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa60435d640 1 -- 172.21.15.70:0/1676764224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f4009a20 con 0x7fa5fc091aa0 2024-09-13T15:42:11.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa5ee7fc640 1 -- 172.21.15.70:0/1676764224 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000cdf0 con 0x7fa5fc091aa0 2024-09-13T15:42:11.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.276+0000 7fa60435d640 1 -- 172.21.15.70:0/1676764224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5fc08f690 con 0x7fa5fc091aa0 2024-09-13T15:42:11.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.277+0000 7fa5ee7fc640 1 -- 172.21.15.70:0/1676764224 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fa5f0005970 con 0x7fa5fc091aa0 2024-09-13T15:42:11.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.277+0000 7fa5ee7fc640 1 -- 172.21.15.70:0/1676764224 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fa5f0006020 con 0x7fa5fc091aa0 2024-09-13T15:42:11.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.277+0000 7fa60435d640 1 -- 172.21.15.70:0/1676764224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5fc05ed70 con 0x7fa5fc091aa0 2024-09-13T15:42:11.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.279+0000 7fa5ee7fc640 1 -- 172.21.15.70:0/1676764224 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fa5f0021060 con 0x7fa5fc091aa0 2024-09-13T15:42:11.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.347+0000 7fa60435d640 1 -- 172.21.15.70:0/1676764224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ba4d90f9-6972-45df-800f-92ad808b06ec", "id": 1} v 0) v1 -- 0x7fa5fc08f910 con 0x7fa5fc091aa0 2024-09-13T15:42:11.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.349+0000 7fa5ee7fc640 1 -- 172.21.15.70:0/1676764224 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ba4d90f9-6972-45df-800f-92ad808b06ec", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa5f0017070 con 0x7fa5fc091aa0 2024-09-13T15:42:11.352 INFO:teuthology.orchestra.run.smithi070.stdout:1 2024-09-13T15:42:11.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.350+0000 7fa60435d640 1 -- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5fc091aa0 msgr2=0x7fa5fc08e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.350+0000 7fa60435d640 1 --2- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5fc091aa0 0x7fa5fc08e6d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000ec80 tx=0x7fa5f00091d0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.350+0000 7fa60435d640 1 -- 172.21.15.70:0/1676764224 shutdown_connections 2024-09-13T15:42:11.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.350+0000 7fa60435d640 1 --2- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5fc1270d0 0x7fa5fc08f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.350+0000 7fa60435d640 1 --2- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa5fc094970 0x7fa5fc08ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.350+0000 7fa60435d640 1 --2- 172.21.15.70:0/1676764224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5fc091aa0 0x7fa5fc08e6d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.350+0000 7fa60435d640 1 -- 172.21.15.70:0/1676764224 >> 172.21.15.70:0/1676764224 conn(0x7fa5fc095a60 msgr2=0x7fa5fc090c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:11.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.350+0000 7fa60435d640 1 -- 172.21.15.70:0/1676764224 shutdown_connections 2024-09-13T15:42:11.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.350+0000 7fa60435d640 1 -- 172.21.15.70:0/1676764224 wait complete. 2024-09-13T15:42:11.362 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new b74d8eb5-3082-4d69-a755-7f5f92dec5b1 2 2024-09-13T15:42:11.413 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:11.412+0000 7fb67c63bec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T15:42:11.414 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:11.413+0000 7ffa221beec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T15:42:11.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.453+0000 7fe54aea8640 1 Processor -- start 2024-09-13T15:42:11.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.453+0000 7fe54aea8640 1 -- start start 2024-09-13T15:42:11.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.453+0000 7fe54aea8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe544063120 0x7fe54412f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.453+0000 7fe54aea8640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544096560 0x7fe544131990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.453+0000 7fe54aea8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe544063cd0 con 0x7fe544095a80 2024-09-13T15:42:11.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.453+0000 7fe54aea8640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54406ebd0 con 0x7fe544063120 2024-09-13T15:42:11.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.453+0000 7fe54aea8640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe54406f310 con 0x7fe544096560 2024-09-13T15:42:11.456 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.454+0000 7fe543fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe544063120 0x7fe54412f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.454+0000 7fe549445640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544096560 0x7fe544131990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.454+0000 7fe543fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe544063120 0x7fe54412f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53548/0 (socket says 172.21.15.70:53548) 2024-09-13T15:42:11.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.454+0000 7fe543fff640 1 -- 172.21.15.70:0/1027308895 learned_addr learned my addr 172.21.15.70:0/1027308895 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:11.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.454+0000 7fe549445640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544096560 0x7fe544131990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:34152/0 (socket says 172.21.15.70:34152) 2024-09-13T15:42:11.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.454+0000 7fe548c44640 1 --1- 172.21.15.70:0/1027308895 >> v1:172.21.15.70:6789/0 conn(0x7fe544095a80 0x7fe544095e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40802/0 (socket says 172.21.15.70:40802) 2024-09-13T15:42:11.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.454+0000 7fe549445640 1 -- 172.21.15.70:0/1027308895 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544096560 msgr2=0x7fe544131990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T15:42:11.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.454+0000 7fe549445640 1 -- 172.21.15.70:0/1027308895 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544096560 msgr2=0x7fe544131990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:42:11.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.454+0000 7fe549445640 1 --2- 172.21.15.70:0/1027308895 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544096560 0x7fe544131990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:11.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.454+0000 7fe549445640 1 --2- 172.21.15.70:0/1027308895 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544096560 0x7fe544131990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:42:11.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.454+0000 7fe543fff640 1 -- 172.21.15.70:0/1027308895 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544096560 msgr2=0x7fe544131990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T15:42:11.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.454+0000 7fe543fff640 1 --2- 172.21.15.70:0/1027308895 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544096560 0x7fe544131990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.454+0000 7fe543fff640 1 -- 172.21.15.70:0/1027308895 >> v1:172.21.15.70:6789/0 conn(0x7fe544095a80 legacy=0x7fe544095e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.457 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.454+0000 7fe543fff640 1 -- 172.21.15.70:0/1027308895 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe544063900 con 0x7fe544063120 2024-09-13T15:42:11.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.454+0000 7fe5437fe640 1 -- 172.21.15.70:0/1027308895 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835386734 0 0) 0x7fe544063cd0 con 0x7fe544095a80 2024-09-13T15:42:11.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.455+0000 7fe543fff640 1 --2- 172.21.15.70:0/1027308895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544063120 0x7fe54412f440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe530009910 tx=0x7fe5300091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f0f96c6f4b4d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:11.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.455+0000 7fe5437fe640 1 -- 172.21.15.70:0/1027308895 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53002ad20 con 0x7fe544063120 2024-09-13T15:42:11.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.455+0000 7fe5437fe640 1 -- 172.21.15.70:0/1027308895 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53002ae80 con 0x7fe544063120 2024-09-13T15:42:11.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.455+0000 7fe5437fe640 1 -- 172.21.15.70:0/1027308895 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530032670 con 0x7fe544063120 2024-09-13T15:42:11.461 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.455+0000 7fe54aea8640 1 -- 172.21.15.70:0/1027308895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544063120 msgr2=0x7fe54412f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.461 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.455+0000 7fe54aea8640 1 --2- 172.21.15.70:0/1027308895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544063120 0x7fe54412f440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe530009910 tx=0x7fe5300091d0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.461 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.456+0000 7fe54aea8640 1 -- 172.21.15.70:0/1027308895 shutdown_connections 2024-09-13T15:42:11.461 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.456+0000 7fe54aea8640 1 --2- 172.21.15.70:0/1027308895 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544096560 0x7fe544131990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.461 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.456+0000 7fe54aea8640 1 --2- 172.21.15.70:0/1027308895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544063120 0x7fe54412f440 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.461 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.456+0000 7fe54aea8640 1 -- 172.21.15.70:0/1027308895 >> 172.21.15.70:0/1027308895 conn(0x7fe544096b70 msgr2=0x7fe544096f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:11.461 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.456+0000 7fe54aea8640 1 -- 172.21.15.70:0/1027308895 shutdown_connections 2024-09-13T15:42:11.461 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.456+0000 7fe54aea8640 1 -- 172.21.15.70:0/1027308895 wait complete. 2024-09-13T15:42:11.461 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.456+0000 7fe54aea8640 1 Processor -- start 2024-09-13T15:42:11.461 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe54aea8640 1 -- start start 2024-09-13T15:42:11.461 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe54aea8640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe544063120 0x7fe54411f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe54aea8640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544095a80 0x7fe54411f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe54aea8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544096560 0x7fe5441392d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe54aea8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe544121980 con 0x7fe544096560 2024-09-13T15:42:11.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe54aea8640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe544099970 con 0x7fe544063120 2024-09-13T15:42:11.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe54aea8640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe544120450 con 0x7fe544095a80 2024-09-13T15:42:11.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe548c44640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe544063120 0x7fe54411f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe543fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544095a80 0x7fe54411f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe548c44640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe544063120 0x7fe54411f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:36332/0 (socket says 172.21.15.70:36332) 2024-09-13T15:42:11.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe548c44640 1 -- 172.21.15.70:0/2338798498 learned_addr learned my addr 172.21.15.70:0/2338798498 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:11.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe549445640 1 --2- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544096560 0x7fe5441392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe543fff640 1 -- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544095a80 msgr2=0x7fe54411f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T15:42:11.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe543fff640 1 -- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544095a80 msgr2=0x7fe54411f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T15:42:11.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe543fff640 1 --2- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544095a80 0x7fe54411f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:11.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe543fff640 1 --2- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544095a80 0x7fe54411f8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:42:11.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe548c44640 1 -- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544095a80 msgr2=0x7fe54411f8c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:42:11.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe548c44640 1 --2- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544095a80 0x7fe54411f8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe548c44640 1 -- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544096560 msgr2=0x7fe5441392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe548c44640 1 --2- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544096560 0x7fe5441392d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe548c44640 1 -- 172.21.15.70:0/2338798498 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe530009450 con 0x7fe544063120 2024-09-13T15:42:11.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.457+0000 7fe549445640 1 --2- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544096560 0x7fe5441392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:11.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.458+0000 7fe548c44640 1 --2- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe544063120 0x7fe54411f380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c5e0 tx=0x7fe538007d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:11.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.458+0000 7fe540ff9640 1 -- 172.21.15.70:0/2338798498 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538007300 con 0x7fe544063120 2024-09-13T15:42:11.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.458+0000 7fe540ff9640 1 -- 172.21.15.70:0/2338798498 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe538007480 con 0x7fe544063120 2024-09-13T15:42:11.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.458+0000 7fe54aea8640 1 -- 172.21.15.70:0/2338798498 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe544122160 con 0x7fe544063120 2024-09-13T15:42:11.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.458+0000 7fe540ff9640 1 -- 172.21.15.70:0/2338798498 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5380049f0 con 0x7fe544063120 2024-09-13T15:42:11.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.458+0000 7fe54aea8640 1 -- 172.21.15.70:0/2338798498 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54411fe00 con 0x7fe544063120 2024-09-13T15:42:11.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.458+0000 7fe540ff9640 1 -- 172.21.15.70:0/2338798498 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fe538012440 con 0x7fe544063120 2024-09-13T15:42:11.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.458+0000 7fe540ff9640 1 -- 172.21.15.70:0/2338798498 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fe538007630 con 0x7fe544063120 2024-09-13T15:42:11.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.459+0000 7fe54aea8640 1 -- 172.21.15.70:0/2338798498 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54405ed70 con 0x7fe544063120 2024-09-13T15:42:11.465 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.461+0000 7fe540ff9640 1 -- 172.21.15.70:0/2338798498 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fe538018070 con 0x7fe544063120 2024-09-13T15:42:11.469 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:11.468+0000 7ffa221beec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T15:42:11.469 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:11.468+0000 7fb67c63bec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T15:42:11.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.528+0000 7fe54aea8640 1 -- 172.21.15.70:0/2338798498 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b74d8eb5-3082-4d69-a755-7f5f92dec5b1", "id": 2} v 0) v1 -- 0x7fe54411fff0 con 0x7fe544063120 2024-09-13T15:42:11.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.531+0000 7fe540ff9640 1 -- 172.21.15.70:0/2338798498 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b74d8eb5-3082-4d69-a755-7f5f92dec5b1", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe538004e20 con 0x7fe544063120 2024-09-13T15:42:11.532 INFO:teuthology.orchestra.run.smithi070.stdout:2 2024-09-13T15:42:11.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.532+0000 7fe54aea8640 1 -- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe544063120 msgr2=0x7fe54411f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.532+0000 7fe54aea8640 1 --2- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe544063120 0x7fe54411f380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe53800c5e0 tx=0x7fe538007d00 comp rx=0 tx=0).stop 2024-09-13T15:42:11.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.532+0000 7fe54aea8640 1 -- 172.21.15.70:0/2338798498 shutdown_connections 2024-09-13T15:42:11.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.532+0000 7fe54aea8640 1 --2- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe544096560 0x7fe5441392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.532+0000 7fe54aea8640 1 --2- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe544095a80 0x7fe54411f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.532+0000 7fe54aea8640 1 --2- 172.21.15.70:0/2338798498 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe544063120 0x7fe54411f380 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.532+0000 7fe54aea8640 1 -- 172.21.15.70:0/2338798498 >> 172.21.15.70:0/2338798498 conn(0x7fe544096b70 msgr2=0x7fe544093570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:11.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.532+0000 7fe54aea8640 1 -- 172.21.15.70:0/2338798498 shutdown_connections 2024-09-13T15:42:11.534 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.532+0000 7fe54aea8640 1 -- 172.21.15.70:0/2338798498 wait complete. 2024-09-13T15:42:11.544 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 02835b7d-e2cf-4ef8-87ba-b225593689d3 3 2024-09-13T15:42:11.594 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:11.593+0000 7fb67c63bec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T15:42:11.594 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:11.593+0000 7ffa221beec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T15:42:11.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.637+0000 7fea13d5d640 1 Processor -- start 2024-09-13T15:42:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.637+0000 7fea13d5d640 1 -- start start 2024-09-13T15:42:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea13d5d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea0c095af0 0x7fea0c095ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea13d5d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 0x7fea0c12e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea13d5d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0c12e7d0 0x7fea0c130bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea13d5d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea0c063a90 con 0x7fea0c095af0 2024-09-13T15:42:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea13d5d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c063cc0 con 0x7fea0c12e7d0 2024-09-13T15:42:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea13d5d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c063880 con 0x7fea0c096400 2024-09-13T15:42:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea11af9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea0c095af0 0x7fea0c095ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea11af9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea0c095af0 0x7fea0c095ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53560/0 (socket says 172.21.15.70:53560) 2024-09-13T15:42:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea11af9640 1 -- 172.21.15.70:0/1910121445 learned_addr learned my addr 172.21.15.70:0/1910121445 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea112f8640 1 --2- 172.21.15.70:0/1910121445 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 0x7fea0c12e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea122fa640 1 --2- 172.21.15.70:0/1910121445 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0c12e7d0 0x7fea0c130bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea112f8640 1 -- 172.21.15.70:0/1910121445 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 msgr2=0x7fea0c12e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T15:42:11.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea112f8640 1 -- 172.21.15.70:0/1910121445 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 msgr2=0x7fea0c12e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:42:11.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea112f8640 1 --2- 172.21.15.70:0/1910121445 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 0x7fea0c12e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:11.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea112f8640 1 --2- 172.21.15.70:0/1910121445 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 0x7fea0c12e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:42:11.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea122fa640 1 -- 172.21.15.70:0/1910121445 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 msgr2=0x7fea0c12e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T15:42:11.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea122fa640 1 --2- 172.21.15.70:0/1910121445 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 0x7fea0c12e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea122fa640 1 -- 172.21.15.70:0/1910121445 >> v2:172.21.15.70:3300/0 conn(0x7fea0c095af0 msgr2=0x7fea0c095ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:11.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea122fa640 1 --2- 172.21.15.70:0/1910121445 >> v2:172.21.15.70:3300/0 conn(0x7fea0c095af0 0x7fea0c095ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea122fa640 1 -- 172.21.15.70:0/1910121445 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c063460 con 0x7fea0c12e7d0 2024-09-13T15:42:11.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.638+0000 7fea11af9640 1 --2- 172.21.15.70:0/1910121445 >> v2:172.21.15.70:3300/0 conn(0x7fea0c095af0 0x7fea0c095ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:11.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.639+0000 7fea122fa640 1 --2- 172.21.15.70:0/1910121445 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0c12e7d0 0x7fea0c130bb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fea04009e60 tx=0x7fea04009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4b0dd789c7971b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:11.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.639+0000 7fea10af7640 1 -- 172.21.15.70:0/1910121445 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04007550 con 0x7fea0c12e7d0 2024-09-13T15:42:11.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.639+0000 7fea10af7640 1 -- 172.21.15.70:0/1910121445 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0402d050 con 0x7fea0c12e7d0 2024-09-13T15:42:11.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.639+0000 7fea10af7640 1 -- 172.21.15.70:0/1910121445 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04007920 con 0x7fea0c12e7d0 2024-09-13T15:42:11.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.640+0000 7fea13d5d640 1 -- 172.21.15.70:0/1910121445 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0c12e7d0 msgr2=0x7fea0c130bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.640+0000 7fea13d5d640 1 --2- 172.21.15.70:0/1910121445 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0c12e7d0 0x7fea0c130bb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fea04009e60 tx=0x7fea04009780 comp rx=0 tx=0).stop 2024-09-13T15:42:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.640+0000 7fea13d5d640 1 -- 172.21.15.70:0/1910121445 shutdown_connections 2024-09-13T15:42:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.640+0000 7fea13d5d640 1 --2- 172.21.15.70:0/1910121445 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0c12e7d0 0x7fea0c130bb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.640+0000 7fea13d5d640 1 --2- 172.21.15.70:0/1910121445 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 0x7fea0c12e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.640+0000 7fea13d5d640 1 --2- 172.21.15.70:0/1910121445 >> v2:172.21.15.70:3300/0 conn(0x7fea0c095af0 0x7fea0c095ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.640+0000 7fea13d5d640 1 -- 172.21.15.70:0/1910121445 >> 172.21.15.70:0/1910121445 conn(0x7fea0c09ad10 msgr2=0x7fea0c096be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.640+0000 7fea13d5d640 1 -- 172.21.15.70:0/1910121445 shutdown_connections 2024-09-13T15:42:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.640+0000 7fea13d5d640 1 -- 172.21.15.70:0/1910121445 wait complete. 2024-09-13T15:42:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea13d5d640 1 Processor -- start 2024-09-13T15:42:11.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea13d5d640 1 -- start start 2024-09-13T15:42:11.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea13d5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea0c095af0 0x7fea0c138fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea13d5d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 0x7fea0c139510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea13d5d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0c12e7d0 0x7fea0c11f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea13d5d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c08d5b0 con 0x7fea0c095af0 2024-09-13T15:42:11.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea13d5d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c08d720 con 0x7fea0c12e7d0 2024-09-13T15:42:11.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea13d5d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c121190 con 0x7fea0c096400 2024-09-13T15:42:11.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea112f8640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 0x7fea0c139510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea122fa640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0c12e7d0 0x7fea0c11f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea112f8640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 0x7fea0c139510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:34196/0 (socket says 172.21.15.70:34196) 2024-09-13T15:42:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea112f8640 1 -- 172.21.15.70:0/3369850453 learned_addr learned my addr 172.21.15.70:0/3369850453 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea11af9640 1 --2- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea0c095af0 0x7fea0c138fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea112f8640 1 -- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 msgr2=0x7fea0c139510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T15:42:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea112f8640 1 -- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 msgr2=0x7fea0c139510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T15:42:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea112f8640 1 --2- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 0x7fea0c139510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.641+0000 7fea112f8640 1 --2- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 0x7fea0c139510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:42:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.642+0000 7fea11af9640 1 -- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 msgr2=0x7fea0c139510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:42:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.642+0000 7fea11af9640 1 --2- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 0x7fea0c139510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.642+0000 7fea11af9640 1 -- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0c12e7d0 msgr2=0x7fea0c11f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.642+0000 7fea11af9640 1 --2- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0c12e7d0 0x7fea0c11f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.642+0000 7fea11af9640 1 -- 172.21.15.70:0/3369850453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea08009450 con 0x7fea0c095af0 2024-09-13T15:42:11.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.642+0000 7fea122fa640 1 --2- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0c12e7d0 0x7fea0c11f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:11.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.642+0000 7fea11af9640 1 --2- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea0c095af0 0x7fea0c138fd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fea0800ea20 tx=0x7fea080091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:11.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.642+0000 7fea01ffb640 1 -- 172.21.15.70:0/3369850453 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08009680 con 0x7fea0c095af0 2024-09-13T15:42:11.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.642+0000 7fea01ffb640 1 -- 172.21.15.70:0/3369850453 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0800ca10 con 0x7fea0c095af0 2024-09-13T15:42:11.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.642+0000 7fea13d5d640 1 -- 172.21.15.70:0/3369850453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea04009a20 con 0x7fea0c095af0 2024-09-13T15:42:11.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.642+0000 7fea01ffb640 1 -- 172.21.15.70:0/3369850453 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0800cdd0 con 0x7fea0c095af0 2024-09-13T15:42:11.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.642+0000 7fea13d5d640 1 -- 172.21.15.70:0/3369850453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0c120620 con 0x7fea0c095af0 2024-09-13T15:42:11.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.642+0000 7fea01ffb640 1 -- 172.21.15.70:0/3369850453 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fea08005970 con 0x7fea0c095af0 2024-09-13T15:42:11.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.642+0000 7fea01ffb640 1 -- 172.21.15.70:0/3369850453 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7fea08013e40 con 0x7fea0c095af0 2024-09-13T15:42:11.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.644+0000 7fea13d5d640 1 -- 172.21.15.70:0/3369850453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0c05ed70 con 0x7fea0c095af0 2024-09-13T15:42:11.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.645+0000 7fea01ffb640 1 -- 172.21.15.70:0/3369850453 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fea0803a7a0 con 0x7fea0c095af0 2024-09-13T15:42:11.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.713+0000 7fea13d5d640 1 -- 172.21.15.70:0/3369850453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "02835b7d-e2cf-4ef8-87ba-b225593689d3", "id": 3} v 0) v1 -- 0x7fea0c070610 con 0x7fea0c095af0 2024-09-13T15:42:11.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.715+0000 7fea01ffb640 1 -- 172.21.15.70:0/3369850453 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "02835b7d-e2cf-4ef8-87ba-b225593689d3", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fea08029030 con 0x7fea0c095af0 2024-09-13T15:42:11.717 INFO:teuthology.orchestra.run.smithi070.stdout:3 2024-09-13T15:42:11.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.716+0000 7fea13d5d640 1 -- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea0c095af0 msgr2=0x7fea0c138fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.716+0000 7fea13d5d640 1 --2- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea0c095af0 0x7fea0c138fd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fea0800ea20 tx=0x7fea080091d0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.716+0000 7fea13d5d640 1 -- 172.21.15.70:0/3369850453 shutdown_connections 2024-09-13T15:42:11.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.716+0000 7fea13d5d640 1 --2- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea0c12e7d0 0x7fea0c11f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.716+0000 7fea13d5d640 1 --2- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea0c096400 0x7fea0c139510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.716+0000 7fea13d5d640 1 --2- 172.21.15.70:0/3369850453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea0c095af0 0x7fea0c138fd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.716+0000 7fea13d5d640 1 -- 172.21.15.70:0/3369850453 >> 172.21.15.70:0/3369850453 conn(0x7fea0c09ad10 msgr2=0x7fea0c0988d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:11.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.717+0000 7fea13d5d640 1 -- 172.21.15.70:0/3369850453 shutdown_connections 2024-09-13T15:42:11.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.717+0000 7fea13d5d640 1 -- 172.21.15.70:0/3369850453 wait complete. 2024-09-13T15:42:11.727 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T15:42:11.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T15:42:11.730 INFO:tasks.ceph.osd.0:Started 2024-09-13T15:42:11.730 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T15:42:11.730 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T15:42:11.732 INFO:tasks.ceph.osd.1:Started 2024-09-13T15:42:11.732 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T15:42:11.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T15:42:11.735 INFO:tasks.ceph.osd.2:Started 2024-09-13T15:42:11.735 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T15:42:11.735 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T15:42:11.737 INFO:tasks.ceph.osd.3:Started 2024-09-13T15:42:11.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T15:42:11.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.879+0000 7fa323754640 1 Processor -- start 2024-09-13T15:42:11.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.879+0000 7fa323754640 1 -- start start 2024-09-13T15:42:11.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.880+0000 7fa323754640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa31c096b40 0x7fa31c08f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.880+0000 7fa323754640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa31c08f820 0x7fa31c08fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.880+0000 7fa323754640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31c0912e0 con 0x7fa31c12c2b0 2024-09-13T15:42:11.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.880+0000 7fa323754640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31c090850 con 0x7fa31c08f820 2024-09-13T15:42:11.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.880+0000 7fa323754640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c090170 con 0x7fa31c096b40 2024-09-13T15:42:11.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.880+0000 7fa3214f0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa31c096b40 0x7fa31c08f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.880+0000 7fa321cf1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa31c12c2b0 0x7fa31c12e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40838/0 (socket says 172.21.15.70:40838) 2024-09-13T15:42:11.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.880+0000 7fa320cef640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa31c08f820 0x7fa31c08fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.880+0000 7fa321cf1640 1 -- 172.21.15.70:0/316870659 learned_addr learned my addr 172.21.15.70:0/316870659 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:42:11.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.880+0000 7fa3214f0640 1 -- 172.21.15.70:0/316870659 >> v2:172.21.15.70:3300/0 conn(0x7fa31c08f820 msgr2=0x7fa31c08fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:11.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.880+0000 7fa3214f0640 1 --2- 172.21.15.70:0/316870659 >> v2:172.21.15.70:3300/0 conn(0x7fa31c08f820 0x7fa31c08fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.880+0000 7fa3214f0640 1 -- 172.21.15.70:0/316870659 >> v1:172.21.15.70:6789/0 conn(0x7fa31c12c2b0 legacy=0x7fa31c12e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.880+0000 7fa3214f0640 1 -- 172.21.15.70:0/316870659 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c097730 con 0x7fa31c096b40 2024-09-13T15:42:11.884 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.880+0000 7fa320cef640 1 --2- 172.21.15.70:0/316870659 >> v2:172.21.15.70:3300/0 conn(0x7fa31c08f820 0x7fa31c08fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:11.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.881+0000 7fa3214f0640 1 --2- 172.21.15.70:0/316870659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa31c096b40 0x7fa31c08f2e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa318009850 tx=0x7fa3180090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7f97018127a1cf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:11.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.881+0000 7fa313fff640 1 -- 172.21.15.70:0/316870659 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31802ad00 con 0x7fa31c096b40 2024-09-13T15:42:11.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.881+0000 7fa313fff640 1 -- 172.21.15.70:0/316870659 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31802ae60 con 0x7fa31c096b40 2024-09-13T15:42:11.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.881+0000 7fa313fff640 1 -- 172.21.15.70:0/316870659 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318032670 con 0x7fa31c096b40 2024-09-13T15:42:11.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.882+0000 7fa323754640 1 -- 172.21.15.70:0/316870659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa31c096b40 msgr2=0x7fa31c08f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.882+0000 7fa323754640 1 --2- 172.21.15.70:0/316870659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa31c096b40 0x7fa31c08f2e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa318009850 tx=0x7fa3180090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.882+0000 7fa323754640 1 -- 172.21.15.70:0/316870659 shutdown_connections 2024-09-13T15:42:11.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.882+0000 7fa323754640 1 --2- 172.21.15.70:0/316870659 >> v2:172.21.15.70:3300/0 conn(0x7fa31c08f820 0x7fa31c08fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.882+0000 7fa323754640 1 --2- 172.21.15.70:0/316870659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa31c096b40 0x7fa31c08f2e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.882+0000 7fa323754640 1 -- 172.21.15.70:0/316870659 >> 172.21.15.70:0/316870659 conn(0x7fa31c097c30 msgr2=0x7fa31c098030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:11.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.882+0000 7fa323754640 1 -- 172.21.15.70:0/316870659 shutdown_connections 2024-09-13T15:42:11.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.882+0000 7fa323754640 1 -- 172.21.15.70:0/316870659 wait complete. 2024-09-13T15:42:11.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa323754640 1 Processor -- start 2024-09-13T15:42:11.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa323754640 1 -- start start 2024-09-13T15:42:11.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa323754640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa31c08f820 0x7fa31c12b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa323754640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa31c096b40 0x7fa31c12bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa323754640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa31c12c2b0 0x7fa31c088180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:11.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa323754640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c08b480 con 0x7fa31c08f820 2024-09-13T15:42:11.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa323754640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c08b850 con 0x7fa31c12c2b0 2024-09-13T15:42:11.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa323754640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c08a4a0 con 0x7fa31c096b40 2024-09-13T15:42:11.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa321cf1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa31c12c2b0 0x7fa31c088180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa321cf1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa31c12c2b0 0x7fa31c088180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:36414/0 (socket says 172.21.15.70:36414) 2024-09-13T15:42:11.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa321cf1640 1 -- 172.21.15.70:0/1882563338 learned_addr learned my addr 172.21.15.70:0/1882563338 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:11.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa320cef640 1 --2- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa31c096b40 0x7fa31c12bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa3214f0640 1 --2- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa31c08f820 0x7fa31c12b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:11.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa320cef640 1 -- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa31c096b40 msgr2=0x7fa31c12bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T15:42:11.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa320cef640 1 -- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa31c096b40 msgr2=0x7fa31c12bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T15:42:11.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa320cef640 1 --2- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa31c096b40 0x7fa31c12bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:11.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.883+0000 7fa320cef640 1 --2- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa31c096b40 0x7fa31c12bd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:42:11.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.884+0000 7fa3214f0640 1 -- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa31c096b40 msgr2=0x7fa31c12bd40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:42:11.890 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.884+0000 7fa3214f0640 1 --2- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa31c096b40 0x7fa31c12bd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.884+0000 7fa3214f0640 1 -- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa31c12c2b0 msgr2=0x7fa31c088180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.884+0000 7fa3214f0640 1 --2- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa31c12c2b0 0x7fa31c088180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.884+0000 7fa3214f0640 1 -- 172.21.15.70:0/1882563338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c009390 con 0x7fa31c08f820 2024-09-13T15:42:11.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.884+0000 7fa3214f0640 1 --2- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa31c08f820 0x7fa31c12b800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa31c06aa10 tx=0x7fa318032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:11.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.884+0000 7fa3117fa640 1 -- 172.21.15.70:0/1882563338 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318031780 con 0x7fa31c08f820 2024-09-13T15:42:11.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.884+0000 7fa3117fa640 1 -- 172.21.15.70:0/1882563338 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa318031900 con 0x7fa31c08f820 2024-09-13T15:42:11.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.884+0000 7fa323754640 1 -- 172.21.15.70:0/1882563338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa318009390 con 0x7fa31c08f820 2024-09-13T15:42:11.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.884+0000 7fa3117fa640 1 -- 172.21.15.70:0/1882563338 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318031cc0 con 0x7fa31c08f820 2024-09-13T15:42:11.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.884+0000 7fa323754640 1 -- 172.21.15.70:0/1882563338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c0886c0 con 0x7fa31c08f820 2024-09-13T15:42:11.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.884+0000 7fa3117fa640 1 -- 172.21.15.70:0/1882563338 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fa318039040 con 0x7fa31c08f820 2024-09-13T15:42:11.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.884+0000 7fa3117fa640 1 -- 172.21.15.70:0/1882563338 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fa318031e20 con 0x7fa31c08f820 2024-09-13T15:42:11.891 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.885+0000 7fa323754640 1 -- 172.21.15.70:0/1882563338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31c05ed70 con 0x7fa31c08f820 2024-09-13T15:42:11.892 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.889+0000 7fa3117fa640 1 -- 172.21.15.70:0/1882563338 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fa31803f070 con 0x7fa31c08f820 2024-09-13T15:42:11.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.946+0000 7fa323754640 1 -- 172.21.15.70:0/1882563338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa31c070450 con 0x7fa31c08f820 2024-09-13T15:42:11.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.947+0000 7fa3117fa640 1 -- 172.21.15.70:0/1882563338 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4108 (secure 0 0 0) 0x7fa318037070 con 0x7fa31c08f820 2024-09-13T15:42:11.948 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:42:11.949 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":6,"fsid":"45bbcdad-f4fb-4639-ab42-ccda8de151dd","created":"2024-09-13T15:42:09.863982+0000","modified":"2024-09-13T15:42:11.714506+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T15:42:11.714506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"265c6f1d-08a7-4c43-b2a6-2417ab814825","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ba4d90f9-6972-45df-800f-92ad808b06ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b74d8eb5-3082-4d69-a755-7f5f92dec5b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"02835b7d-e2cf-4ef8-87ba-b225593689d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T15:42:11.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.947+0000 7fa323754640 1 -- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa31c08f820 msgr2=0x7fa31c12b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:11.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.947+0000 7fa323754640 1 --2- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa31c08f820 0x7fa31c12b800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa31c06aa10 tx=0x7fa318032000 comp rx=0 tx=0).stop 2024-09-13T15:42:11.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.947+0000 7fa323754640 1 -- 172.21.15.70:0/1882563338 shutdown_connections 2024-09-13T15:42:11.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.947+0000 7fa323754640 1 --2- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa31c12c2b0 0x7fa31c088180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.948+0000 7fa323754640 1 --2- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa31c096b40 0x7fa31c12bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.948+0000 7fa323754640 1 --2- 172.21.15.70:0/1882563338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa31c08f820 0x7fa31c12b800 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:11.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.948+0000 7fa323754640 1 -- 172.21.15.70:0/1882563338 >> 172.21.15.70:0/1882563338 conn(0x7fa31c097c30 msgr2=0x7fa31c12e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:11.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.948+0000 7fa323754640 1 -- 172.21.15.70:0/1882563338 shutdown_connections 2024-09-13T15:42:11.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:11.948+0000 7fa323754640 1 -- 172.21.15.70:0/1882563338 wait complete. 2024-09-13T15:42:11.959 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T15:42:11.959 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T15:42:11.965 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:11.964+0000 7fb67c63bec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T15:42:11.965 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:11.964+0000 7ffa221beec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T15:42:12.016 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:12.015+0000 7fb67c63bec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T15:42:12.017 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:12.016+0000 7ffa221beec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T15:42:12.101 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:12.100+0000 7fb67c63bec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T15:42:12.101 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:12.100+0000 7ffa221beec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T15:42:12.178 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:12.176+0000 7fb67c63bec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T15:42:12.178 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:12.176+0000 7ffa221beec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T15:42:12.224 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:12.222+0000 7fb67c63bec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T15:42:12.224 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:12.222+0000 7ffa221beec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T15:42:12.306 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:12.304+0000 7fb67c63bec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T15:42:12.306 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:12.305+0000 7ffa221beec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T15:42:12.344 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-13T15:42:12.343+0000 7f82f2830440 -1 Falling back to public interface 2024-09-13T15:42:12.352 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-13T15:42:12.351+0000 7ff384f27440 -1 Falling back to public interface 2024-09-13T15:42:12.359 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:12.358+0000 7fb67c63bec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T15:42:12.360 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:12.358+0000 7ffa221beec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T15:42:12.360 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-13T15:42:12.359+0000 7f11bd017440 -1 Falling back to public interface 2024-09-13T15:42:12.362 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T15:42:12.363 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-13T15:42:12.361+0000 7fee0aa40440 -1 Falling back to public interface 2024-09-13T15:42:12.406 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:12.404+0000 7ffa221beec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T15:42:12.406 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:12.405+0000 7fb67c63bec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T15:42:12.450 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.448+0000 7f938599d640 1 Processor -- start 2024-09-13T15:42:12.451 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.448+0000 7f938599d640 1 -- start start 2024-09-13T15:42:12.451 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.448+0000 7f938599d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f938005db50 0x7f93800953b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:12.451 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.448+0000 7f938599d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f93800958f0 0x7f9380095ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:12.451 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.448+0000 7f938599d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938006f6b0 con 0x7f938012eb60 2024-09-13T15:42:12.451 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.448+0000 7f938599d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938006b420 con 0x7f93800958f0 2024-09-13T15:42:12.451 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.448+0000 7f938599d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938006ef70 con 0x7f938005db50 2024-09-13T15:42:12.451 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.448+0000 7f937e7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f93800958f0 0x7f9380095ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:12.451 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.448+0000 7f937effd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f938005db50 0x7f93800953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:12.451 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.448+0000 7f937f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f938012eb60 0x7f9380130f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40840/0 (socket says 172.21.15.70:40840) 2024-09-13T15:42:12.451 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.448+0000 7f937f7fe640 1 -- 172.21.15.70:0/1483635284 learned_addr learned my addr 172.21.15.70:0/1483635284 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:42:12.451 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.449+0000 7f937dffb640 1 -- 172.21.15.70:0/1483635284 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153251928 0 0) 0x7f938006f6b0 con 0x7f938012eb60 2024-09-13T15:42:12.451 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.449+0000 7f937dffb640 1 -- 172.21.15.70:0/1483635284 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93680034e0 con 0x7f938012eb60 2024-09-13T15:42:12.452 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.449+0000 7f937e7fc640 1 -- 172.21.15.70:0/1483635284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f938005db50 msgr2=0x7f93800953b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:12.452 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.449+0000 7f937e7fc640 1 --2- 172.21.15.70:0/1483635284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f938005db50 0x7f93800953b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:12.452 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.449+0000 7f937e7fc640 1 -- 172.21.15.70:0/1483635284 >> v1:172.21.15.70:6789/0 conn(0x7f938012eb60 legacy=0x7f9380130f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:12.452 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.449+0000 7f937e7fc640 1 -- 172.21.15.70:0/1483635284 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9380096220 con 0x7f93800958f0 2024-09-13T15:42:12.452 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.449+0000 7f937effd640 1 --2- 172.21.15.70:0/1483635284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f938005db50 0x7f93800953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:12.456 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.450+0000 7f937e7fc640 1 --2- 172.21.15.70:0/1483635284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93800958f0 0x7f9380095ce0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9374009880 tx=0x7f93740090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90bad1096f2d6fb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:12.456 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.450+0000 7f937dffb640 1 -- 172.21.15.70:0/1483635284 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937402ac50 con 0x7f93800958f0 2024-09-13T15:42:12.456 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.450+0000 7f937dffb640 1 -- 172.21.15.70:0/1483635284 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937402adb0 con 0x7f93800958f0 2024-09-13T15:42:12.456 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.450+0000 7f937dffb640 1 -- 172.21.15.70:0/1483635284 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374032670 con 0x7f93800958f0 2024-09-13T15:42:12.456 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.451+0000 7f938599d640 1 -- 172.21.15.70:0/1483635284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93800958f0 msgr2=0x7f9380095ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:12.456 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.451+0000 7f938599d640 1 --2- 172.21.15.70:0/1483635284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93800958f0 0x7f9380095ce0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9374009880 tx=0x7f93740090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:12.456 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.451+0000 7f938599d640 1 -- 172.21.15.70:0/1483635284 shutdown_connections 2024-09-13T15:42:12.456 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.451+0000 7f938599d640 1 --2- 172.21.15.70:0/1483635284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93800958f0 0x7f9380095ce0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:12.456 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.451+0000 7f938599d640 1 --2- 172.21.15.70:0/1483635284 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f938005db50 0x7f93800953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:12.456 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.451+0000 7f938599d640 1 -- 172.21.15.70:0/1483635284 >> 172.21.15.70:0/1483635284 conn(0x7f93800964a0 msgr2=0x7f93800968a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:12.456 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.451+0000 7f938599d640 1 -- 172.21.15.70:0/1483635284 shutdown_connections 2024-09-13T15:42:12.456 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.451+0000 7f938599d640 1 -- 172.21.15.70:0/1483635284 wait complete. 2024-09-13T15:42:12.457 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.452+0000 7f938599d640 1 Processor -- start 2024-09-13T15:42:12.457 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.452+0000 7f938599d640 1 -- start start 2024-09-13T15:42:12.457 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f938599d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f938005db50 0x7f9380139250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:12.457 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f938599d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f93800958f0 0x7f9380139790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:12.457 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f938599d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f938012eb60 0x7f9380139cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:12.457 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f938599d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9380092b80 con 0x7f938012eb60 2024-09-13T15:42:12.457 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f938599d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9380093120 con 0x7f938005db50 2024-09-13T15:42:12.457 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f938599d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f938011f8a0 con 0x7f93800958f0 2024-09-13T15:42:12.457 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937effd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f938005db50 0x7f9380139250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:12.457 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f938012eb60 0x7f9380139cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:12.457 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937e7fc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f93800958f0 0x7f9380139790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:12.457 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937effd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f938005db50 0x7f9380139250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:36428/0 (socket says 172.21.15.70:36428) 2024-09-13T15:42:12.457 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f938012eb60 0x7f9380139cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53628/0 (socket says 172.21.15.70:53628) 2024-09-13T15:42:12.458 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937effd640 1 -- 172.21.15.70:0/543544631 learned_addr learned my addr 172.21.15.70:0/543544631 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:12.458 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937e7fc640 1 -- 172.21.15.70:0/543544631 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f93800958f0 msgr2=0x7f9380139790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T15:42:12.458 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937e7fc640 1 -- 172.21.15.70:0/543544631 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f93800958f0 msgr2=0x7f9380139790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T15:42:12.458 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937e7fc640 1 --2- 172.21.15.70:0/543544631 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f93800958f0 0x7f9380139790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-13T15:42:12.458 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937e7fc640 1 --2- 172.21.15.70:0/543544631 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f93800958f0 0x7f9380139790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:42:12.458 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937effd640 1 -- 172.21.15.70:0/543544631 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f93800958f0 msgr2=0x7f9380139790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:42:12.458 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937effd640 1 --2- 172.21.15.70:0/543544631 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f93800958f0 0x7f9380139790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:12.458 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937effd640 1 -- 172.21.15.70:0/543544631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f938012eb60 msgr2=0x7f9380139cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:12.458 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937effd640 1 --2- 172.21.15.70:0/543544631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f938012eb60 0x7f9380139cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:12.458 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937effd640 1 -- 172.21.15.70:0/543544631 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9370009390 con 0x7f938005db50 2024-09-13T15:42:12.458 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f937effd640 1 --2- 172.21.15.70:0/543544631 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f938005db50 0x7f9380139250 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f937000ec80 tx=0x7f93700090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:12.458 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f9366ffd640 1 -- 172.21.15.70:0/543544631 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937000eec0 con 0x7f938005db50 2024-09-13T15:42:12.458 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f9366ffd640 1 -- 172.21.15.70:0/543544631 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937000cbc0 con 0x7f938005db50 2024-09-13T15:42:12.458 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f938599d640 1 -- 172.21.15.70:0/543544631 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9374009390 con 0x7f938005db50 2024-09-13T15:42:12.459 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.453+0000 7f9366ffd640 1 -- 172.21.15.70:0/543544631 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9370013470 con 0x7f938005db50 2024-09-13T15:42:12.459 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.454+0000 7f938599d640 1 -- 172.21.15.70:0/543544631 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9380091970 con 0x7f938005db50 2024-09-13T15:42:12.459 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.454+0000 7f9366ffd640 1 -- 172.21.15.70:0/543544631 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f9370005970 con 0x7f938005db50 2024-09-13T15:42:12.459 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.454+0000 7f9366ffd640 1 -- 172.21.15.70:0/543544631 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f9370006020 con 0x7f938005db50 2024-09-13T15:42:12.459 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.455+0000 7f938599d640 1 -- 172.21.15.70:0/543544631 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9380070ed0 con 0x7f938005db50 2024-09-13T15:42:12.459 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.457+0000 7f9366ffd640 1 -- 172.21.15.70:0/543544631 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f9380070ed0 con 0x7f938005db50 2024-09-13T15:42:12.501 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:12.500+0000 7fb67c63bec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T15:42:12.503 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:12.502+0000 7ffa221beec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T15:42:12.529 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.528+0000 7f938599d640 1 -- 172.21.15.70:0/543544631 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f938005ed70 con 0x7f938005db50 2024-09-13T15:42:12.529 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.528+0000 7f9366ffd640 1 -- 172.21.15.70:0/543544631 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4108 (secure 0 0 0) 0x7f9370021060 con 0x7f938005db50 2024-09-13T15:42:12.530 INFO:teuthology.misc.health.smithi070.stdout: 2024-09-13T15:42:12.530 INFO:teuthology.misc.health.smithi070.stdout:{"epoch":6,"fsid":"45bbcdad-f4fb-4639-ab42-ccda8de151dd","created":"2024-09-13T15:42:09.863982+0000","modified":"2024-09-13T15:42:11.714506+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T15:42:11.714506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"265c6f1d-08a7-4c43-b2a6-2417ab814825","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ba4d90f9-6972-45df-800f-92ad808b06ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b74d8eb5-3082-4d69-a755-7f5f92dec5b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"02835b7d-e2cf-4ef8-87ba-b225593689d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T15:42:12.531 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.529+0000 7f938599d640 1 -- 172.21.15.70:0/543544631 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f938005db50 msgr2=0x7f9380139250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:12.531 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.529+0000 7f938599d640 1 --2- 172.21.15.70:0/543544631 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f938005db50 0x7f9380139250 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f937000ec80 tx=0x7f93700090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:12.531 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.529+0000 7f938599d640 1 -- 172.21.15.70:0/543544631 shutdown_connections 2024-09-13T15:42:12.531 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.529+0000 7f938599d640 1 --2- 172.21.15.70:0/543544631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f938012eb60 0x7f9380139cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:12.531 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.529+0000 7f938599d640 1 --2- 172.21.15.70:0/543544631 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f93800958f0 0x7f9380139790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:12.531 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.529+0000 7f938599d640 1 --2- 172.21.15.70:0/543544631 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f938005db50 0x7f9380139250 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:12.531 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.529+0000 7f938599d640 1 -- 172.21.15.70:0/543544631 >> 172.21.15.70:0/543544631 conn(0x7f93800964a0 msgr2=0x7f938012df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:12.531 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.529+0000 7f938599d640 1 -- 172.21.15.70:0/543544631 shutdown_connections 2024-09-13T15:42:12.531 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:12.529+0000 7f938599d640 1 -- 172.21.15.70:0/543544631 wait complete. 2024-09-13T15:42:12.539 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-13T15:42:12.552 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:12.551+0000 7fb67c63bec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T15:42:12.552 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:12.551+0000 7ffa221beec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T15:42:12.603 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:12.602+0000 7fb67c63bec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T15:42:12.603 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:12.602+0000 7ffa221beec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T15:42:12.672 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:12.671+0000 7fb67c63bec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T15:42:12.672 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:12.671+0000 7ffa221beec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T15:42:12.715 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:12.714+0000 7ffa221beec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T15:42:12.715 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:12.714+0000 7fb67c63bec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T15:42:12.757 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:12.756+0000 7ffa221beec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T15:42:12.758 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:12.756+0000 7fb67c63bec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T15:42:12.965 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:42:12.964+0000 7fb67c63bec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T15:42:12.965 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:12.964+0000 7ffa221beec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T15:42:13.976 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:13.975+0000 7ffa0b064640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T15:42:13.977 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:42:13.975+0000 7ffa0b064640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T15:42:15.828 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-13T15:42:15.827+0000 7f82f2830440 -1 osd.1 0 log_to_monitors true 2024-09-13T15:42:15.855 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-13T15:42:15.853+0000 7ff384f27440 -1 osd.0 0 log_to_monitors true 2024-09-13T15:42:15.883 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-13T15:42:15.882+0000 7f11bd017440 -1 osd.3 0 log_to_monitors true 2024-09-13T15:42:15.884 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-13T15:42:15.883+0000 7fee0aa40440 -1 osd.2 0 log_to_monitors true 2024-09-13T15:42:18.942 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T15:42:19.073 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-13T15:42:19.072+0000 7f11b77f6640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T15:42:19.079 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.078+0000 7f2681f1d640 1 Processor -- start 2024-09-13T15:42:19.079 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.078+0000 7f2681f1d640 1 -- start start 2024-09-13T15:42:19.081 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.079+0000 7f2681f1d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267c06a9e0 0x7f267c06adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:19.081 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.079+0000 7f2681f1d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f267c06b2f0 0x7f267c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:19.081 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.079+0000 7f2681f1d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f267c06cb80 con 0x7f267c132510 2024-09-13T15:42:19.081 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.079+0000 7f2681f1d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f267c06dc10 con 0x7f267c06b2f0 2024-09-13T15:42:19.082 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.079+0000 7f2681f1d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267c06d840 con 0x7f267c06a9e0 2024-09-13T15:42:19.082 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.079+0000 7f267affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f267c06b2f0 0x7f267c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:19.082 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.079+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267c06a9e0 0x7f267c06adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:19.082 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.079+0000 7f267affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f267c06b2f0 0x7f267c06b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57720/0 (socket says 172.21.15.70:57720) 2024-09-13T15:42:19.082 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.079+0000 7f267affd640 1 -- 172.21.15.70:0/2612808128 learned_addr learned my addr 172.21.15.70:0/2612808128 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:19.082 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.079+0000 7f267b7fe640 1 -- 172.21.15.70:0/2612808128 >> v2:172.21.15.70:3300/0 conn(0x7f267c06b2f0 msgr2=0x7f267c06b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:19.082 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.079+0000 7f267b7fe640 1 --2- 172.21.15.70:0/2612808128 >> v2:172.21.15.70:3300/0 conn(0x7f267c06b2f0 0x7f267c06b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:19.082 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.080+0000 7f267b7fe640 1 -- 172.21.15.70:0/2612808128 >> v1:172.21.15.70:6789/0 conn(0x7f267c132510 legacy=0x7f267c1348f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T15:42:19.082 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.080+0000 7f267b7fe640 1 -- 172.21.15.70:0/2612808128 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267c06d060 con 0x7f267c06a9e0 2024-09-13T15:42:19.083 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-13T15:42:19.080+0000 7f11b07e8640 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T15:42:19.084 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.081+0000 7f267b7fe640 1 --2- 172.21.15.70:0/2612808128 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267c06a9e0 0x7f267c06adb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2674009ab0 tx=0x7f26740090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1226d0b51c4314e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:19.084 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.081+0000 7f267a7fc640 1 -- 172.21.15.70:0/2612808128 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267402ac30 con 0x7f267c06a9e0 2024-09-13T15:42:19.084 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.081+0000 7f267a7fc640 1 -- 172.21.15.70:0/2612808128 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267402ad90 con 0x7f267c06a9e0 2024-09-13T15:42:19.084 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.082+0000 7f267a7fc640 1 -- 172.21.15.70:0/2612808128 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2674032670 con 0x7f267c06a9e0 2024-09-13T15:42:19.084 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.082+0000 7f2681f1d640 1 -- 172.21.15.70:0/2612808128 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267c06a9e0 msgr2=0x7f267c06adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:19.085 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.082+0000 7f2681f1d640 1 --2- 172.21.15.70:0/2612808128 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267c06a9e0 0x7f267c06adb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2674009ab0 tx=0x7f26740090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:19.085 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.082+0000 7f2681f1d640 1 -- 172.21.15.70:0/2612808128 shutdown_connections 2024-09-13T15:42:19.085 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.082+0000 7f2681f1d640 1 --2- 172.21.15.70:0/2612808128 >> v2:172.21.15.70:3300/0 conn(0x7f267c06b2f0 0x7f267c06b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:19.085 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.082+0000 7f2681f1d640 1 --2- 172.21.15.70:0/2612808128 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267c06a9e0 0x7f267c06adb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:19.085 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.082+0000 7f2681f1d640 1 -- 172.21.15.70:0/2612808128 >> 172.21.15.70:0/2612808128 conn(0x7f267c0996c0 msgr2=0x7f267c099ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:19.085 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.082+0000 7f2681f1d640 1 -- 172.21.15.70:0/2612808128 shutdown_connections 2024-09-13T15:42:19.085 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.082+0000 7f2681f1d640 1 -- 172.21.15.70:0/2612808128 wait complete. 2024-09-13T15:42:19.085 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-13T15:42:19.083+0000 7f82ecff6640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T15:42:19.088 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.083+0000 7f2681f1d640 1 Processor -- start 2024-09-13T15:42:19.088 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.083+0000 7f2681f1d640 1 -- start start 2024-09-13T15:42:19.088 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.084+0000 7f2681f1d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f267c06a9e0 0x7f267c130640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:19.088 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.084+0000 7f2681f1d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267c06b2f0 0x7f267c130b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:19.089 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.084+0000 7f2681f1d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f267c132510 0x7f267c1310c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:19.089 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.084+0000 7f2681f1d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c132220 con 0x7f267c06a9e0 2024-09-13T15:42:19.089 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.084+0000 7f2681f1d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267c1316f0 con 0x7f267c06b2f0 2024-09-13T15:42:19.089 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.084+0000 7f2681f1d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f267c11d2c0 con 0x7f267c132510 2024-09-13T15:42:19.089 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.085+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f267c06a9e0 0x7f267c130640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:19.089 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.085+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f267c06a9e0 0x7f267c130640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57726/0 (socket says 172.21.15.70:57726) 2024-09-13T15:42:19.089 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.085+0000 7f267b7fe640 1 -- 172.21.15.70:0/3249681272 learned_addr learned my addr 172.21.15.70:0/3249681272 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:19.090 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.085+0000 7f267affd640 1 --2- 172.21.15.70:0/3249681272 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267c06b2f0 0x7f267c130b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:19.090 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.085+0000 7f267b7fe640 1 -- 172.21.15.70:0/3249681272 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f267c132510 msgr2=0x7f267c1310c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:19.090 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.085+0000 7f267b7fe640 1 --2- 172.21.15.70:0/3249681272 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f267c132510 0x7f267c1310c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:19.090 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.085+0000 7f267b7fe640 1 -- 172.21.15.70:0/3249681272 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267c06b2f0 msgr2=0x7f267c130b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:19.090 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.085+0000 7f267b7fe640 1 --2- 172.21.15.70:0/3249681272 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267c06b2f0 0x7f267c130b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:19.090 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.085+0000 7f267b7fe640 1 -- 172.21.15.70:0/3249681272 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2674009390 con 0x7f267c06a9e0 2024-09-13T15:42:19.091 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.086+0000 7f267b7fe640 1 --2- 172.21.15.70:0/3249681272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f267c06a9e0 0x7f267c130640 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2674009360 tx=0x7f2674032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:19.091 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-13T15:42:19.087+0000 7f82e5fe8640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T15:42:19.093 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.088+0000 7f26637fe640 1 -- 172.21.15.70:0/3249681272 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267403c620 con 0x7f267c06a9e0 2024-09-13T15:42:19.093 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.089+0000 7f26637fe640 1 -- 172.21.15.70:0/3249681272 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267403c780 con 0x7f267c06a9e0 2024-09-13T15:42:19.093 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.090+0000 7f2681f1d640 1 -- 172.21.15.70:0/3249681272 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267c11d910 con 0x7f267c06a9e0 2024-09-13T15:42:19.094 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.090+0000 7f26637fe640 1 -- 172.21.15.70:0/3249681272 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267403cb20 con 0x7f267c06a9e0 2024-09-13T15:42:19.094 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.090+0000 7f2681f1d640 1 -- 172.21.15.70:0/3249681272 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267c13d9f0 con 0x7f267c06a9e0 2024-09-13T15:42:19.094 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.091+0000 7f2681f1d640 1 -- 172.21.15.70:0/3249681272 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267c070ed0 con 0x7f267c06a9e0 2024-09-13T15:42:19.094 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.091+0000 7f26637fe640 1 -- 172.21.15.70:0/3249681272 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 58495+0+0 (secure 0 0 0) 0x7f2674039070 con 0x7f267c06a9e0 2024-09-13T15:42:19.094 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.091+0000 7f26637fe640 1 -- 172.21.15.70:0/3249681272 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1509+0+0 (secure 0 0 0) 0x7f2674083050 con 0x7f267c06a9e0 2024-09-13T15:42:19.098 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.097+0000 7f26637fe640 1 -- 172.21.15.70:0/3249681272 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f267c070ed0 con 0x7f267c06a9e0 2024-09-13T15:42:19.115 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-13T15:42:19.114+0000 7fee051f6640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T15:42:19.120 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-13T15:42:19.119+0000 7fedfe1e8640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T15:42:19.123 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-13T15:42:19.122+0000 7ff37f9f6640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T15:42:19.128 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-13T15:42:19.126+0000 7ff378884640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T15:42:19.235 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.234+0000 7f2681f1d640 1 -- 172.21.15.70:0/3249681272 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f267c070ed0 con 0x7f267c06a9e0 2024-09-13T15:42:19.236 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.234+0000 7f26637fe640 1 -- 172.21.15.70:0/3249681272 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+4108 (secure 0 0 0) 0x7f267c070ed0 con 0x7f267c06a9e0 2024-09-13T15:42:19.236 INFO:teuthology.misc.health.smithi070.stdout: 2024-09-13T15:42:19.236 INFO:teuthology.misc.health.smithi070.stdout:{"epoch":8,"fsid":"45bbcdad-f4fb-4639-ab42-ccda8de151dd","created":"2024-09-13T15:42:09.863982+0000","modified":"2024-09-13T15:42:17.674115+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T15:42:11.714506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"265c6f1d-08a7-4c43-b2a6-2417ab814825","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ba4d90f9-6972-45df-800f-92ad808b06ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b74d8eb5-3082-4d69-a755-7f5f92dec5b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"02835b7d-e2cf-4ef8-87ba-b225593689d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T15:42:19.237 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.236+0000 7f2681f1d640 1 -- 172.21.15.70:0/3249681272 >> v1:172.21.15.70:6816/32242 conn(0x7f266804cae0 legacy=0x7f266804ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:19.238 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.236+0000 7f2681f1d640 1 -- 172.21.15.70:0/3249681272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f267c06a9e0 msgr2=0x7f267c130640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:19.238 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.236+0000 7f2681f1d640 1 --2- 172.21.15.70:0/3249681272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f267c06a9e0 0x7f267c130640 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2674009360 tx=0x7f2674032000 comp rx=0 tx=0).stop 2024-09-13T15:42:19.238 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.236+0000 7f2681f1d640 1 -- 172.21.15.70:0/3249681272 shutdown_connections 2024-09-13T15:42:19.238 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.236+0000 7f2681f1d640 1 --2- 172.21.15.70:0/3249681272 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f267c132510 0x7f267c1310c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:19.238 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.236+0000 7f2681f1d640 1 --2- 172.21.15.70:0/3249681272 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f267c06b2f0 0x7f267c130b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:19.238 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.236+0000 7f2681f1d640 1 --2- 172.21.15.70:0/3249681272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f267c06a9e0 0x7f267c130640 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:19.238 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.236+0000 7f2681f1d640 1 -- 172.21.15.70:0/3249681272 >> 172.21.15.70:0/3249681272 conn(0x7f267c0996c0 msgr2=0x7f267c10f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:19.238 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.236+0000 7f2681f1d640 1 -- 172.21.15.70:0/3249681272 shutdown_connections 2024-09-13T15:42:19.238 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:19.236+0000 7f2681f1d640 1 -- 172.21.15.70:0/3249681272 wait complete. 2024-09-13T15:42:19.246 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-13T15:42:25.649 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T15:42:25.749 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.746+0000 7f1eaab9d640 1 Processor -- start 2024-09-13T15:42:25.749 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.746+0000 7f1eaab9d640 1 -- start start 2024-09-13T15:42:25.749 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.746+0000 7f1eaab9d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1ea406e360 0x7f1ea4127be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:25.749 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.746+0000 7f1eaab9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea4128120 0x7f1ea4128570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:25.749 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.746+0000 7f1eaab9d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea4110190 con 0x7f1ea406a7b0 2024-09-13T15:42:25.750 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.746+0000 7f1eaab9d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea410ffc0 con 0x7f1ea4128120 2024-09-13T15:42:25.750 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.746+0000 7f1eaab9d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea4110a40 con 0x7f1ea406e360 2024-09-13T15:42:25.750 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.747+0000 7f1ea8939640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1ea406e360 0x7f1ea4127be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:25.750 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.747+0000 7f1ea3fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea4128120 0x7f1ea4128570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:25.750 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.747+0000 7f1ea913a640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1ea406a7b0 0x7f1ea406ac30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38654/0 (socket says 172.21.15.70:38654) 2024-09-13T15:42:25.750 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.747+0000 7f1ea913a640 1 -- 172.21.15.70:0/2386592092 learned_addr learned my addr 172.21.15.70:0/2386592092 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:42:25.750 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.747+0000 7f1ea8939640 1 -- 172.21.15.70:0/2386592092 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea4128120 msgr2=0x7f1ea4128570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:25.750 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.747+0000 7f1ea8939640 1 --2- 172.21.15.70:0/2386592092 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea4128120 0x7f1ea4128570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:25.750 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.747+0000 7f1ea37fe640 1 -- 172.21.15.70:0/2386592092 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394466862 0 0) 0x7f1ea4110190 con 0x7f1ea406a7b0 2024-09-13T15:42:25.750 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.747+0000 7f1ea8939640 1 -- 172.21.15.70:0/2386592092 >> v1:172.21.15.70:6789/0 conn(0x7f1ea406a7b0 legacy=0x7f1ea406ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:25.750 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.747+0000 7f1ea8939640 1 -- 172.21.15.70:0/2386592092 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea4110600 con 0x7f1ea406e360 2024-09-13T15:42:25.750 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.747+0000 7f1ea3fff640 1 --2- 172.21.15.70:0/2386592092 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea4128120 0x7f1ea4128570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:25.754 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.748+0000 7f1ea8939640 1 --2- 172.21.15.70:0/2386592092 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1ea406e360 0x7f1ea4127be0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009880 tx=0x7f1e980090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbafb919ac87a3d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:25.754 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.748+0000 7f1ea37fe640 1 -- 172.21.15.70:0/2386592092 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9802ac50 con 0x7f1ea406e360 2024-09-13T15:42:25.754 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.748+0000 7f1ea37fe640 1 -- 172.21.15.70:0/2386592092 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9802adb0 con 0x7f1ea406e360 2024-09-13T15:42:25.754 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.748+0000 7f1ea37fe640 1 -- 172.21.15.70:0/2386592092 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98032670 con 0x7f1ea406e360 2024-09-13T15:42:25.754 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.748+0000 7f1eaab9d640 1 -- 172.21.15.70:0/2386592092 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1ea406e360 msgr2=0x7f1ea4127be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:25.754 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.748+0000 7f1eaab9d640 1 --2- 172.21.15.70:0/2386592092 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1ea406e360 0x7f1ea4127be0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009880 tx=0x7f1e980090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:25.754 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.748+0000 7f1eaab9d640 1 -- 172.21.15.70:0/2386592092 shutdown_connections 2024-09-13T15:42:25.754 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.748+0000 7f1eaab9d640 1 --2- 172.21.15.70:0/2386592092 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea4128120 0x7f1ea4128570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:25.754 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.748+0000 7f1eaab9d640 1 --2- 172.21.15.70:0/2386592092 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1ea406e360 0x7f1ea4127be0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:25.754 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.748+0000 7f1eaab9d640 1 -- 172.21.15.70:0/2386592092 >> 172.21.15.70:0/2386592092 conn(0x7f1ea40996c0 msgr2=0x7f1ea4099ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:25.754 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.748+0000 7f1eaab9d640 1 -- 172.21.15.70:0/2386592092 shutdown_connections 2024-09-13T15:42:25.754 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.748+0000 7f1eaab9d640 1 -- 172.21.15.70:0/2386592092 wait complete. 2024-09-13T15:42:25.755 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.749+0000 7f1eaab9d640 1 Processor -- start 2024-09-13T15:42:25.755 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.749+0000 7f1eaab9d640 1 -- start start 2024-09-13T15:42:25.755 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1eaab9d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1ea406a7b0 0x7f1ea4123990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:25.755 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1eaab9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea406e360 0x7f1ea4123ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:25.755 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1eaab9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea4128120 0x7f1ea4124410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:25.755 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1eaab9d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea406d9a0 con 0x7f1ea4128120 2024-09-13T15:42:25.755 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1eaab9d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea406c230 con 0x7f1ea406e360 2024-09-13T15:42:25.755 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1eaab9d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea412f6a0 con 0x7f1ea406a7b0 2024-09-13T15:42:25.755 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1ea8939640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1ea406a7b0 0x7f1ea4123990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:25.755 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1ea3fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea406e360 0x7f1ea4123ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:25.755 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1ea913a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea4128120 0x7f1ea4124410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:25.755 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1ea8939640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1ea406a7b0 0x7f1ea4123990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41372/0 (socket says 172.21.15.70:41372) 2024-09-13T15:42:25.755 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1ea8939640 1 -- 172.21.15.70:0/3476851472 learned_addr learned my addr 172.21.15.70:0/3476851472 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:25.756 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1ea913a640 1 -- 172.21.15.70:0/3476851472 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1ea406a7b0 msgr2=0x7f1ea4123990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:25.756 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1ea913a640 1 --2- 172.21.15.70:0/3476851472 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1ea406a7b0 0x7f1ea4123990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:25.756 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1ea913a640 1 -- 172.21.15.70:0/3476851472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea406e360 msgr2=0x7f1ea4123ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:25.756 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1ea913a640 1 --2- 172.21.15.70:0/3476851472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea406e360 0x7f1ea4123ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:25.756 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1ea913a640 1 -- 172.21.15.70:0/3476851472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e90009390 con 0x7f1ea4128120 2024-09-13T15:42:25.756 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1ea3fff640 1 --2- 172.21.15.70:0/3476851472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea406e360 0x7f1ea4123ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:25.756 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.750+0000 7f1ea913a640 1 --2- 172.21.15.70:0/3476851472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea4128120 0x7f1ea4124410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00cbe0 tx=0x7f1e9c0019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:25.756 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.751+0000 7f1ea0ff9640 1 -- 172.21.15.70:0/3476851472 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c00ce00 con 0x7f1ea4128120 2024-09-13T15:42:25.756 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.751+0000 7f1ea0ff9640 1 -- 172.21.15.70:0/3476851472 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9c00a850 con 0x7f1ea4128120 2024-09-13T15:42:25.756 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.751+0000 7f1eaab9d640 1 -- 172.21.15.70:0/3476851472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e98009390 con 0x7f1ea4128120 2024-09-13T15:42:25.756 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.751+0000 7f1ea0ff9640 1 -- 172.21.15.70:0/3476851472 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c00ac90 con 0x7f1ea4128120 2024-09-13T15:42:25.756 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.751+0000 7f1eaab9d640 1 -- 172.21.15.70:0/3476851472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea406d120 con 0x7f1ea4128120 2024-09-13T15:42:25.756 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.752+0000 7f1eaab9d640 1 -- 172.21.15.70:0/3476851472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea4070ed0 con 0x7f1ea4128120 2024-09-13T15:42:25.756 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.752+0000 7f1ea0ff9640 1 -- 172.21.15.70:0/3476851472 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f1e9c007300 con 0x7f1ea4128120 2024-09-13T15:42:25.757 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.752+0000 7f1ea0ff9640 1 -- 172.21.15.70:0/3476851472 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2455+0+0 (secure 0 0 0) 0x7f1e9c0630b0 con 0x7f1ea4128120 2024-09-13T15:42:25.757 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.756+0000 7f1ea0ff9640 1 -- 172.21.15.70:0/3476851472 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f1ea4070ed0 con 0x7f1ea4128120 2024-09-13T15:42:25.894 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.889+0000 7f1eaab9d640 1 -- 172.21.15.70:0/3476851472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ea4070ed0 con 0x7f1ea4128120 2024-09-13T15:42:25.894 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.889+0000 7f1ea0ff9640 1 -- 172.21.15.70:0/3476851472 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+6543 (secure 0 0 0) 0x7f1ea4070ed0 con 0x7f1ea4128120 2024-09-13T15:42:25.894 INFO:teuthology.misc.health.smithi070.stdout: 2024-09-13T15:42:25.895 INFO:teuthology.misc.health.smithi070.stdout:{"epoch":12,"fsid":"45bbcdad-f4fb-4639-ab42-ccda8de151dd","created":"2024-09-13T15:42:09.863982+0000","modified":"2024-09-13T15:42:23.698385+0000","last_up_change":"2024-09-13T15:42:20.070768+0000","last_in_change":"2024-09-13T15:42:11.714506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T15:42:21.000499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"265c6f1d-08a7-4c43-b2a6-2417ab814825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":32759}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":32759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":32759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":32759}]},"public_addr":"172.21.15.70:6804/32759","cluster_addr":"172.21.15.70:6805/32759","heartbeat_back_addr":"172.21.15.70:6807/32759","heartbeat_front_addr":"172.21.15.70:6806/32759","state":["exists","up"]},{"osd":1,"uuid":"ba4d90f9-6972-45df-800f-92ad808b06ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":32758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":32758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":32758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":32758}]},"public_addr":"172.21.15.70:6800/32758","cluster_addr":"172.21.15.70:6801/32758","heartbeat_back_addr":"172.21.15.70:6803/32758","heartbeat_front_addr":"172.21.15.70:6802/32758","state":["exists","up"]},{"osd":2,"uuid":"b74d8eb5-3082-4d69-a755-7f5f92dec5b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":32760}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":32760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":32760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":32760}]},"public_addr":"172.21.15.70:6812/32760","cluster_addr":"172.21.15.70:6813/32760","heartbeat_back_addr":"172.21.15.70:6815/32760","heartbeat_front_addr":"172.21.15.70:6814/32760","state":["exists","up"]},{"osd":3,"uuid":"02835b7d-e2cf-4ef8-87ba-b225593689d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":32769}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":32769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":32769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":32769}]},"public_addr":"172.21.15.70:6808/32769","cluster_addr":"172.21.15.70:6809/32769","heartbeat_back_addr":"172.21.15.70:6811/32769","heartbeat_front_addr":"172.21.15.70:6810/32769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.853420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.807491+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.925567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.883852+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T15:42:25.896 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.891+0000 7f1eaab9d640 1 -- 172.21.15.70:0/3476851472 >> v1:172.21.15.70:6816/32242 conn(0x7f1e8804cb70 legacy=0x7f1e8804f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:25.896 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.891+0000 7f1eaab9d640 1 -- 172.21.15.70:0/3476851472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea4128120 msgr2=0x7f1ea4124410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:25.897 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.891+0000 7f1eaab9d640 1 --2- 172.21.15.70:0/3476851472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea4128120 0x7f1ea4124410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00cbe0 tx=0x7f1e9c0019a0 comp rx=0 tx=0).stop 2024-09-13T15:42:25.897 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.891+0000 7f1eaab9d640 1 -- 172.21.15.70:0/3476851472 shutdown_connections 2024-09-13T15:42:25.897 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.891+0000 7f1eaab9d640 1 --2- 172.21.15.70:0/3476851472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea4128120 0x7f1ea4124410 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:25.897 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.891+0000 7f1eaab9d640 1 --2- 172.21.15.70:0/3476851472 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1ea406e360 0x7f1ea4123ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:25.898 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.891+0000 7f1eaab9d640 1 --2- 172.21.15.70:0/3476851472 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1ea406a7b0 0x7f1ea4123990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:25.898 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.891+0000 7f1eaab9d640 1 -- 172.21.15.70:0/3476851472 >> 172.21.15.70:0/3476851472 conn(0x7f1ea40996c0 msgr2=0x7f1ea4096a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:25.898 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.891+0000 7f1eaab9d640 1 -- 172.21.15.70:0/3476851472 shutdown_connections 2024-09-13T15:42:25.898 INFO:teuthology.misc.health.smithi070.stderr:2024-09-13T15:42:25.891+0000 7f1eaab9d640 1 -- 172.21.15.70:0/3476851472 wait complete. 2024-09-13T15:42:25.902 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-09-13T15:42:25.902 INFO:tasks.ceph:Creating RBD pool 2024-09-13T15:42:25.902 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-13T15:42:25.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.992+0000 7f86f06c9640 1 Processor -- start 2024-09-13T15:42:25.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.992+0000 7f86f06c9640 1 -- start start 2024-09-13T15:42:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86f06c9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f86e8129f10 0x7f86e812a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86f06c9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86e80970e0 0x7f86e8097560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86f06c9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e80998a0 con 0x7f86e812a820 2024-09-13T15:42:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86f06c9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e8098970 con 0x7f86e8129f10 2024-09-13T15:42:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86f06c9640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86e8099110 con 0x7f86e80970e0 2024-09-13T15:42:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86eec66640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86e80970e0 0x7f86e8097560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86ee465640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f86e8129f10 0x7f86e812a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86edc64640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f86e812a820 0x7f86e80969d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38664/0 (socket says 172.21.15.70:38664) 2024-09-13T15:42:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86ee465640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f86e8129f10 0x7f86e812a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43670/0 (socket says 172.21.15.70:43670) 2024-09-13T15:42:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86eec66640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86e80970e0 0x7f86e8097560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41380/0 (socket says 172.21.15.70:41380) 2024-09-13T15:42:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86edc64640 1 -- 172.21.15.70:0/1057281069 learned_addr learned my addr 172.21.15.70:0/1057281069 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:42:25.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86ee465640 1 -- 172.21.15.70:0/1057281069 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86e80970e0 msgr2=0x7f86e8097560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:25.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86ee465640 1 --2- 172.21.15.70:0/1057281069 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86e80970e0 0x7f86e8097560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:25.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86ee465640 1 -- 172.21.15.70:0/1057281069 >> v1:172.21.15.70:6789/0 conn(0x7f86e812a820 legacy=0x7f86e80969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:25.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86ee465640 1 -- 172.21.15.70:0/1057281069 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e809a100 con 0x7f86e8129f10 2024-09-13T15:42:25.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.993+0000 7f86ed463640 1 -- 172.21.15.70:0/1057281069 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281272120 0 0) 0x7f86e80998a0 con 0x7f86e812a820 2024-09-13T15:42:26.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.994+0000 7f86ee465640 1 --2- 172.21.15.70:0/1057281069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e8129f10 0x7f86e812a2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009850 tx=0x7f86e40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b449154583193bb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:26.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.994+0000 7f86ed463640 1 -- 172.21.15.70:0/1057281069 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e402ac80 con 0x7f86e8129f10 2024-09-13T15:42:26.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.994+0000 7f86ed463640 1 -- 172.21.15.70:0/1057281069 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e402ade0 con 0x7f86e8129f10 2024-09-13T15:42:26.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.994+0000 7f86ed463640 1 -- 172.21.15.70:0/1057281069 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4032670 con 0x7f86e8129f10 2024-09-13T15:42:26.004 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.994+0000 7f86f06c9640 1 -- 172.21.15.70:0/1057281069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e8129f10 msgr2=0x7f86e812a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:26.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.994+0000 7f86f06c9640 1 --2- 172.21.15.70:0/1057281069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e8129f10 0x7f86e812a2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009850 tx=0x7f86e40090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:26.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.995+0000 7f86f06c9640 1 -- 172.21.15.70:0/1057281069 shutdown_connections 2024-09-13T15:42:26.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.995+0000 7f86f06c9640 1 --2- 172.21.15.70:0/1057281069 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86e80970e0 0x7f86e8097560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:26.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.995+0000 7f86f06c9640 1 --2- 172.21.15.70:0/1057281069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e8129f10 0x7f86e812a2e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:26.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.995+0000 7f86f06c9640 1 -- 172.21.15.70:0/1057281069 >> 172.21.15.70:0/1057281069 conn(0x7f86e809d8c0 msgr2=0x7f86e809dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:26.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.995+0000 7f86f06c9640 1 -- 172.21.15.70:0/1057281069 shutdown_connections 2024-09-13T15:42:26.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.995+0000 7f86f06c9640 1 -- 172.21.15.70:0/1057281069 wait complete. 2024-09-13T15:42:26.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.995+0000 7f86f06c9640 1 Processor -- start 2024-09-13T15:42:26.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.995+0000 7f86f06c9640 1 -- start start 2024-09-13T15:42:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86f06c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e80970e0 0x7f86e81390f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86f06c9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86e8129f10 0x7f86e8139630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86f06c9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86e812a820 0x7f86e8139b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86f06c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e811e690 con 0x7f86e80970e0 2024-09-13T15:42:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86f06c9640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e811cc70 con 0x7f86e8129f10 2024-09-13T15:42:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86f06c9640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86e811e860 con 0x7f86e812a820 2024-09-13T15:42:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86ee465640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e80970e0 0x7f86e81390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86eec66640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86e812a820 0x7f86e8139b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:26.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86ee465640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e80970e0 0x7f86e81390f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43678/0 (socket says 172.21.15.70:43678) 2024-09-13T15:42:26.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86ee465640 1 -- 172.21.15.70:0/2607001449 learned_addr learned my addr 172.21.15.70:0/2607001449 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:26.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86edc64640 1 --2- 172.21.15.70:0/2607001449 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86e8129f10 0x7f86e8139630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:26.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86ee465640 1 -- 172.21.15.70:0/2607001449 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86e812a820 msgr2=0x7f86e8139b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:26.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86ee465640 1 --2- 172.21.15.70:0/2607001449 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86e812a820 0x7f86e8139b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:26.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86ee465640 1 -- 172.21.15.70:0/2607001449 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86e8129f10 msgr2=0x7f86e8139630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:26.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86ee465640 1 --2- 172.21.15.70:0/2607001449 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86e8129f10 0x7f86e8139630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:26.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86ee465640 1 -- 172.21.15.70:0/2607001449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e4009390 con 0x7f86e80970e0 2024-09-13T15:42:26.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86edc64640 1 --2- 172.21.15.70:0/2607001449 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86e8129f10 0x7f86e8139630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:26.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86eec66640 1 --2- 172.21.15.70:0/2607001449 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86e812a820 0x7f86e8139b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:26.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86ee465640 1 --2- 172.21.15.70:0/2607001449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e80970e0 0x7f86e81390f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f86e402a5b0 tx=0x7f86e4032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:26.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86de7fc640 1 -- 172.21.15.70:0/2607001449 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e403d720 con 0x7f86e80970e0 2024-09-13T15:42:26.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86de7fc640 1 -- 172.21.15.70:0/2607001449 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e403d880 con 0x7f86e80970e0 2024-09-13T15:42:26.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86f06c9640 1 -- 172.21.15.70:0/2607001449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e811ee00 con 0x7f86e80970e0 2024-09-13T15:42:26.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86de7fc640 1 -- 172.21.15.70:0/2607001449 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e403dc20 con 0x7f86e80970e0 2024-09-13T15:42:26.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.996+0000 7f86f06c9640 1 -- 172.21.15.70:0/2607001449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e811bbd0 con 0x7f86e80970e0 2024-09-13T15:42:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.997+0000 7f86f06c9640 1 -- 172.21.15.70:0/2607001449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e811f300 con 0x7f86e80970e0 2024-09-13T15:42:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.997+0000 7f86de7fc640 1 -- 172.21.15.70:0/2607001449 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f86e4039040 con 0x7f86e80970e0 2024-09-13T15:42:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:25.998+0000 7f86de7fc640 1 -- 172.21.15.70:0/2607001449 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2455+0+0 (secure 0 0 0) 0x7f86e4082bf0 con 0x7f86e80970e0 2024-09-13T15:42:26.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.001+0000 7f86de7fc640 1 -- 172.21.15.70:0/2607001449 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f86e405dec0 con 0x7f86e80970e0 2024-09-13T15:42:26.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.131+0000 7f86f06c9640 1 -- 172.21.15.70:0/2607001449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f86e805ed70 con 0x7f86e80970e0 2024-09-13T15:42:26.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.711+0000 7f86de7fc640 1 -- 172.21.15.70:0/2607001449 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) v1 ==== 109+0+0 (secure 0 0 0) 0x7f86e40420a0 con 0x7f86e80970e0 2024-09-13T15:42:26.714 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'rbd' created 2024-09-13T15:42:26.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.716+0000 7f86f06c9640 1 -- 172.21.15.70:0/2607001449 >> v1:172.21.15.70:6816/32242 conn(0x7f86cc04cb70 legacy=0x7f86cc04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:26.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.716+0000 7f86f06c9640 1 -- 172.21.15.70:0/2607001449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e80970e0 msgr2=0x7f86e81390f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:26.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.716+0000 7f86f06c9640 1 --2- 172.21.15.70:0/2607001449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e80970e0 0x7f86e81390f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f86e402a5b0 tx=0x7f86e4032000 comp rx=0 tx=0).stop 2024-09-13T15:42:26.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.716+0000 7f86f06c9640 1 -- 172.21.15.70:0/2607001449 shutdown_connections 2024-09-13T15:42:26.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.716+0000 7f86f06c9640 1 --2- 172.21.15.70:0/2607001449 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86e812a820 0x7f86e8139b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:26.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.716+0000 7f86f06c9640 1 --2- 172.21.15.70:0/2607001449 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86e8129f10 0x7f86e8139630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:26.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.716+0000 7f86f06c9640 1 --2- 172.21.15.70:0/2607001449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e80970e0 0x7f86e81390f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:26.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.716+0000 7f86f06c9640 1 -- 172.21.15.70:0/2607001449 >> 172.21.15.70:0/2607001449 conn(0x7f86e809d8c0 msgr2=0x7f86e8114940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:26.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.716+0000 7f86f06c9640 1 -- 172.21.15.70:0/2607001449 shutdown_connections 2024-09-13T15:42:26.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.716+0000 7f86f06c9640 1 -- 172.21.15.70:0/2607001449 wait complete. 2024-09-13T15:42:26.729 DEBUG:teuthology.orchestra.run.smithi070:> rbd --cluster ceph pool init rbd 2024-09-13T15:42:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.836+0000 7f1bc4e4bb80 1 Processor -- start 2024-09-13T15:42:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.836+0000 7f1bc4e4bb80 1 -- start start 2024-09-13T15:42:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.836+0000 7f1bc4e4bb80 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x555ce72b35f0 0x555ce72b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.836+0000 7f1bc4e4bb80 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ce72b3f00 0x555ce7356b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.836+0000 7f1bc4e4bb80 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x555ce71e70e0 con 0x555ce72b42d0 2024-09-13T15:42:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.836+0000 7f1bc4e4bb80 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ce7220260 con 0x555ce72b3f00 2024-09-13T15:42:26.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.836+0000 7f1bc4e4bb80 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x555ce721bf40 con 0x555ce72b35f0 2024-09-13T15:42:26.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.837+0000 7f1bc264b640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x555ce72b35f0 0x555ce72b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:26.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.837+0000 7f1bbbfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ce72b3f00 0x555ce7356b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:26.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.837+0000 7f1bc2e4c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x555ce72b42d0 0x555ce7359050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38674/0 (socket says 172.21.15.70:38674) 2024-09-13T15:42:26.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.837+0000 7f1bc2e4c640 1 -- 172.21.15.70:0/268040877 learned_addr learned my addr 172.21.15.70:0/268040877 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:42:26.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.837+0000 7f1bbbfff640 1 -- 172.21.15.70:0/268040877 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x555ce72b35f0 msgr2=0x555ce72b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:26.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.837+0000 7f1bbbfff640 1 --2- 172.21.15.70:0/268040877 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x555ce72b35f0 0x555ce72b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:26.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.837+0000 7f1bbbfff640 1 -- 172.21.15.70:0/268040877 >> v1:172.21.15.70:6789/0 conn(0x555ce72b42d0 legacy=0x555ce7359050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:26.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.837+0000 7f1bc1e4a640 1 -- 172.21.15.70:0/268040877 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357842356 0 0) 0x555ce71e70e0 con 0x555ce72b42d0 2024-09-13T15:42:26.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.837+0000 7f1bbbfff640 1 -- 172.21.15.70:0/268040877 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ce71c0270 con 0x555ce72b3f00 2024-09-13T15:42:26.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.837+0000 7f1bc264b640 1 --2- 172.21.15.70:0/268040877 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x555ce72b35f0 0x555ce72b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:26.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.839+0000 7f1bbbfff640 1 --2- 172.21.15.70:0/268040877 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ce72b3f00 0x555ce7356b00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00a680 tx=0x7f1bac045cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec783495ddae08f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:26.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.839+0000 7f1bc1e4a640 1 -- 172.21.15.70:0/268040877 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac04d4e0 con 0x555ce72b3f00 2024-09-13T15:42:26.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.839+0000 7f1bc1e4a640 1 -- 172.21.15.70:0/268040877 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bac04d640 con 0x555ce72b3f00 2024-09-13T15:42:26.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.839+0000 7f1bc1e4a640 1 -- 172.21.15.70:0/268040877 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac04d7d0 con 0x555ce72b3f00 2024-09-13T15:42:26.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.840+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/268040877 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ce72b3f00 msgr2=0x555ce7356b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:26.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.840+0000 7f1bc4e4bb80 1 --2- 172.21.15.70:0/268040877 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ce72b3f00 0x555ce7356b00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00a680 tx=0x7f1bac045cd0 comp rx=0 tx=0).stop 2024-09-13T15:42:26.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.840+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/268040877 shutdown_connections 2024-09-13T15:42:26.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.840+0000 7f1bc4e4bb80 1 --2- 172.21.15.70:0/268040877 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ce72b3f00 0x555ce7356b00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:26.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.840+0000 7f1bc4e4bb80 1 --2- 172.21.15.70:0/268040877 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x555ce72b35f0 0x555ce72b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:26.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.840+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/268040877 >> 172.21.15.70:0/268040877 conn(0x555ce72a8d70 msgr2=0x555ce72ab280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:26.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.840+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/268040877 shutdown_connections 2024-09-13T15:42:26.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.840+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/268040877 wait complete. 2024-09-13T15:42:26.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.840+0000 7f1bc4e4bb80 1 Processor -- start 2024-09-13T15:42:26.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bc4e4bb80 1 -- start start 2024-09-13T15:42:26.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bc4e4bb80 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ce72b35f0 0x555ce72bb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:26.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bc4e4bb80 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x555ce72b3f00 0x555ce72bb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:26.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bc4e4bb80 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ce72b42d0 0x555ce72bbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:26.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bc4e4bb80 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ce71deba0 con 0x555ce72b35f0 2024-09-13T15:42:26.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bc4e4bb80 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x555ce71de840 con 0x555ce72b42d0 2024-09-13T15:42:26.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bc4e4bb80 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x555ce71a17a0 con 0x555ce72b3f00 2024-09-13T15:42:26.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bbbfff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x555ce72b3f00 0x555ce72bb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:26.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bc2e4c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ce72b42d0 0x555ce72bbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:26.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bc264b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ce72b35f0 0x555ce72bb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:26.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bbbfff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x555ce72b3f00 0x555ce72bb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41416/0 (socket says 172.21.15.70:41416) 2024-09-13T15:42:26.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bbbfff640 1 -- 172.21.15.70:0/1440899366 learned_addr learned my addr 172.21.15.70:0/1440899366 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:26.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bbbfff640 1 -- 172.21.15.70:0/1440899366 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ce72b42d0 msgr2=0x555ce72bbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:26.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bbbfff640 1 --2- 172.21.15.70:0/1440899366 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ce72b42d0 0x555ce72bbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:26.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bbbfff640 1 -- 172.21.15.70:0/1440899366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ce72b35f0 msgr2=0x555ce72bb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:26.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bbbfff640 1 --2- 172.21.15.70:0/1440899366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ce72b35f0 0x555ce72bb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:26.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bbbfff640 1 -- 172.21.15.70:0/1440899366 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb4005cf0 con 0x555ce72b3f00 2024-09-13T15:42:26.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.841+0000 7f1bc2e4c640 1 --2- 172.21.15.70:0/1440899366 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ce72b42d0 0x555ce72bbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:26.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.842+0000 7f1bbbfff640 1 --2- 172.21.15.70:0/1440899366 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x555ce72b3f00 0x555ce72bb750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1bac046200 tx=0x7f1bac058000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:26.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.842+0000 7f1bba7fc640 1 -- 172.21.15.70:0/1440899366 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac055440 con 0x555ce72b3f00 2024-09-13T15:42:26.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.842+0000 7f1bba7fc640 1 -- 172.21.15.70:0/1440899366 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bac0555a0 con 0x555ce72b3f00 2024-09-13T15:42:26.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.842+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/1440899366 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bac005cf0 con 0x555ce72b3f00 2024-09-13T15:42:26.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.842+0000 7f1bba7fc640 1 -- 172.21.15.70:0/1440899366 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac055960 con 0x555ce72b3f00 2024-09-13T15:42:26.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.842+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/1440899366 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ce71cd0e0 con 0x555ce72b3f00 2024-09-13T15:42:26.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.843+0000 7f1bba7fc640 1 -- 172.21.15.70:0/1440899366 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f1bac05c070 con 0x555ce72b3f00 2024-09-13T15:42:26.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.843+0000 7f1bba7fc640 1 -- 172.21.15.70:0/1440899366 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2797+0+0 (secure 0 0 0) 0x7f1bac0a59a0 con 0x555ce72b3f00 2024-09-13T15:42:26.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:26.843+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/1440899366 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) v1 -- 0x555ce71c0600 con 0x555ce72b3f00 2024-09-13T15:42:27.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:27.712+0000 7f1bba7fc640 1 -- 172.21.15.70:0/1440899366 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v14) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1bac05a070 con 0x555ce72b3f00 2024-09-13T15:42:27.714 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:27.713+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/1440899366 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x555ce71de630 con 0x555ce72b3f00 2024-09-13T15:42:27.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:27.713+0000 7f1bba7fc640 1 -- 172.21.15.70:0/1440899366 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bac084930 con 0x555ce72b3f00 2024-09-13T15:42:27.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:27.713+0000 7f1bc1649640 1 -- 172.21.15.70:0/1440899366 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=14}) v3 -- 0x7f1ba8002990 con 0x555ce72b3f00 2024-09-13T15:42:27.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:27.714+0000 7f1bba7fc640 1 -- 172.21.15.70:0/1440899366 <== mon.2 v2:172.21.15.70:3302/0 8 ==== osd_map(14..14 src has 1..14) v4 ==== 642+0+0 (secure 0 0 0) 0x7f1bac0846d0 con 0x555ce72b3f00 2024-09-13T15:42:27.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:27.714+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/1440899366 --> v1:172.21.15.70:6808/32769 -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14) v8 -- 0x555ce736b590 con 0x555ce7367a90 2024-09-13T15:42:27.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:27.721+0000 7f1bc2e4c640 1 -- 172.21.15.70:0/1440899366 <== osd.3 v1:172.21.15.70:6808/32769 1 ==== osd_op_reply(1 rbd_trash [create] v14'1 uv1 ondisk = 0) v8 ==== 153+0+0 (unknown 1742540995 0 0) 0x7f1bbc001770 con 0x555ce7367a90 2024-09-13T15:42:27.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:27.724+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/1440899366 --> v1:172.21.15.70:6808/32769 -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e14) v8 -- 0x555ce73b89c0 con 0x555ce7367a90 2024-09-13T15:42:27.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:27.725+0000 7f1bc2e4c640 1 -- 172.21.15.70:0/1440899366 <== osd.3 v1:172.21.15.70:6808/32769 2 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (unknown 173568156 0 0) 0x7f1bbc001770 con 0x555ce7367a90 2024-09-13T15:42:27.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:27.725+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/1440899366 --> v1:172.21.15.70:6808/32769 -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e14) v8 -- 0x555ce73b7ab0 con 0x555ce7367a90 2024-09-13T15:42:27.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:27.725+0000 7f1bc2e4c640 1 -- 172.21.15.70:0/1440899366 <== osd.3 v1:172.21.15.70:6808/32769 3 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (unknown 185109136 0 0) 0x7f1bbc001770 con 0x555ce7367a90 2024-09-13T15:42:27.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:27.725+0000 7f1bc1649640 1 -- 172.21.15.70:0/1440899366 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) v4 -- 0x7f1ba8003d30 con 0x555ce72b3f00 2024-09-13T15:42:28.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:28.713+0000 7f1bba7fc640 1 -- 172.21.15.70:0/1440899366 <== mon.2 v2:172.21.15.70:3302/0 9 ==== pool_op_reply(tid 4 (0) Success v15) v1 ==== 55+0+0 (secure 0 0 0) 0x7f1bac063030 con 0x555ce72b3f00 2024-09-13T15:42:28.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:28.713+0000 7f1bba7fc640 1 -- 172.21.15.70:0/1440899366 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=15}) v3 -- 0x7f1ba4051f20 con 0x555ce72b3f00 2024-09-13T15:42:28.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:28.713+0000 7f1bba7fc640 1 -- 172.21.15.70:0/1440899366 <== mon.2 v2:172.21.15.70:3302/0 10 ==== osd_map(15..15 src has 1..15) v4 ==== 618+0+0 (secure 0 0 0) 0x7f1bac0a4b30 con 0x555ce72b3f00 2024-09-13T15:42:28.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:28.713+0000 7f1bc1649640 1 -- 172.21.15.70:0/1440899366 --> v1:172.21.15.70:6808/32769 -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) v8 -- 0x7f1ba80056b0 con 0x555ce7367a90 2024-09-13T15:42:28.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:28.716+0000 7f1bc2e4c640 1 -- 172.21.15.70:0/1440899366 <== osd.3 v1:172.21.15.70:6808/32769 4 ==== osd_op_reply(5 rbd_info [create,write 0~8] v15'1 uv1 ondisk = 0) v8 ==== 194+0+0 (unknown 913822303 0 0) 0x7f1bbc001770 con 0x555ce7367a90 2024-09-13T15:42:28.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:28.717+0000 7f1bc1649640 1 -- 172.21.15.70:0/1440899366 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v15) v4 -- 0x7f1ba80051c0 con 0x555ce72b3f00 2024-09-13T15:42:29.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.717+0000 7f1bba7fc640 1 -- 172.21.15.70:0/1440899366 <== mon.2 v2:172.21.15.70:3302/0 11 ==== pool_op_reply(tid 6 (0) Success v16) v1 ==== 43+0+0 (secure 0 0 0) 0x7f1bac063020 con 0x555ce72b3f00 2024-09-13T15:42:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.717+0000 7f1bba7fc640 1 -- 172.21.15.70:0/1440899366 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=16}) v3 -- 0x7f1ba4051f20 con 0x555ce72b3f00 2024-09-13T15:42:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.717+0000 7f1bba7fc640 1 -- 172.21.15.70:0/1440899366 <== mon.2 v2:172.21.15.70:3302/0 12 ==== osd_map(16..16 src has 1..16) v4 ==== 646+0+0 (secure 0 0 0) 0x7f1bac055cd0 con 0x555ce72b3f00 2024-09-13T15:42:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.717+0000 7f1bc1649640 1 -- 172.21.15.70:0/1440899366 --> v1:172.21.15.70:6808/32769 -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16) v8 -- 0x7f1ba8005f50 con 0x555ce7367a90 2024-09-13T15:42:29.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.721+0000 7f1bc2e4c640 1 -- 172.21.15.70:0/1440899366 <== osd.3 v1:172.21.15.70:6808/32769 5 ==== osd_op_reply(7 rbd_info [write 0~19] v16'2 uv2 ondisk = 0) v8 ==== 152+0+0 (unknown 2130366450 0 0) 0x7f1bbc001770 con 0x555ce7367a90 2024-09-13T15:42:29.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.721+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/1440899366 >> v1:172.21.15.70:6808/32769 conn(0x555ce7367a90 legacy=0x555ce7369e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.721+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/1440899366 >> v1:172.21.15.70:6816/32242 conn(0x7f1ba404b4a0 legacy=0x7f1ba404d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.721+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/1440899366 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x555ce72b3f00 msgr2=0x555ce72bb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.721+0000 7f1bc4e4bb80 1 --2- 172.21.15.70:0/1440899366 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x555ce72b3f00 0x555ce72bb750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1bac046200 tx=0x7f1bac058000 comp rx=0 tx=0).stop 2024-09-13T15:42:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.721+0000 7f1bc2e4c640 1 -- 172.21.15.70:0/1440899366 reap_dead start 2024-09-13T15:42:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.721+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/1440899366 shutdown_connections 2024-09-13T15:42:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.721+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/1440899366 >> 172.21.15.70:0/1440899366 conn(0x555ce72a8d70 msgr2=0x555ce72c08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.721+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/1440899366 shutdown_connections 2024-09-13T15:42:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.721+0000 7f1bc4e4bb80 1 -- 172.21.15.70:0/1440899366 wait complete. 2024-09-13T15:42:29.726 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T15:42:29.727 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T15:42:29.727 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T15:42:29.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T15:42:29.727 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T15:42:29.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.824+0000 7fb4a335d640 1 Processor -- start 2024-09-13T15:42:29.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.824+0000 7fb4a335d640 1 -- start start 2024-09-13T15:42:29.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.824+0000 7fb4a335d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb49c090a40 0x7fb49c090e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.824+0000 7fb4a335d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb49c1261c0 0x7fb49c1265b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.824+0000 7fb4a335d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb49c090480 con 0x7fb49c126af0 2024-09-13T15:42:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.824+0000 7fb4a335d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb49c092ac0 con 0x7fb49c1261c0 2024-09-13T15:42:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.824+0000 7fb4a335d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c092d10 con 0x7fb49c090a40 2024-09-13T15:42:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.824+0000 7fb4a10f9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb49c090a40 0x7fb49c090e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.824+0000 7fb4a08f8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb49c1261c0 0x7fb49c1265b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.824+0000 7fb4a18fa640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb49c126af0 0x7fb49c12d500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38682/0 (socket says 172.21.15.70:38682) 2024-09-13T15:42:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.824+0000 7fb4a10f9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb49c090a40 0x7fb49c090e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41422/0 (socket says 172.21.15.70:41422) 2024-09-13T15:42:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.824+0000 7fb4a18fa640 1 -- 172.21.15.70:0/1899212852 learned_addr learned my addr 172.21.15.70:0/1899212852 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:42:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.825+0000 7fb4a08f8640 1 -- 172.21.15.70:0/1899212852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb49c090a40 msgr2=0x7fb49c090e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.825+0000 7fb4a08f8640 1 --2- 172.21.15.70:0/1899212852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb49c090a40 0x7fb49c090e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.825+0000 7fb4a08f8640 1 -- 172.21.15.70:0/1899212852 >> v1:172.21.15.70:6789/0 conn(0x7fb49c126af0 legacy=0x7fb49c12d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.825+0000 7fb4a08f8640 1 -- 172.21.15.70:0/1899212852 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c0953c0 con 0x7fb49c1261c0 2024-09-13T15:42:29.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.825+0000 7fb4a10f9640 1 --2- 172.21.15.70:0/1899212852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb49c090a40 0x7fb49c090e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:29.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.825+0000 7fb4a08f8640 1 --2- 172.21.15.70:0/1899212852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49c1261c0 0x7fb49c1265b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb490009850 tx=0x7fb4900090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb382e4cbdf24b06 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:29.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.825+0000 7fb49bfff640 1 -- 172.21.15.70:0/1899212852 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49002ac80 con 0x7fb49c1261c0 2024-09-13T15:42:29.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.825+0000 7fb49bfff640 1 -- 172.21.15.70:0/1899212852 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49002ade0 con 0x7fb49c1261c0 2024-09-13T15:42:29.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.825+0000 7fb49bfff640 1 -- 172.21.15.70:0/1899212852 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490032670 con 0x7fb49c1261c0 2024-09-13T15:42:29.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.826+0000 7fb4a335d640 1 -- 172.21.15.70:0/1899212852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49c1261c0 msgr2=0x7fb49c1265b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:29.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.826+0000 7fb4a335d640 1 --2- 172.21.15.70:0/1899212852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49c1261c0 0x7fb49c1265b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb490009850 tx=0x7fb4900090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.826+0000 7fb4a335d640 1 -- 172.21.15.70:0/1899212852 shutdown_connections 2024-09-13T15:42:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.826+0000 7fb4a335d640 1 --2- 172.21.15.70:0/1899212852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49c1261c0 0x7fb49c1265b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.826+0000 7fb4a335d640 1 --2- 172.21.15.70:0/1899212852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb49c090a40 0x7fb49c090e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.826+0000 7fb4a335d640 1 -- 172.21.15.70:0/1899212852 >> 172.21.15.70:0/1899212852 conn(0x7fb49c0959f0 msgr2=0x7fb49c095df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.826+0000 7fb4a335d640 1 -- 172.21.15.70:0/1899212852 shutdown_connections 2024-09-13T15:42:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.826+0000 7fb4a335d640 1 -- 172.21.15.70:0/1899212852 wait complete. 2024-09-13T15:42:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.827+0000 7fb4a335d640 1 Processor -- start 2024-09-13T15:42:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.827+0000 7fb4a335d640 1 -- start start 2024-09-13T15:42:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.827+0000 7fb4a335d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb49c090a40 0x7fb49c08f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.827+0000 7fb4a335d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49c1261c0 0x7fb49c124140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.827+0000 7fb4a335d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb49c126af0 0x7fb49c124680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a335d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c06af30 con 0x7fb49c1261c0 2024-09-13T15:42:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a335d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c125ba0 con 0x7fb49c126af0 2024-09-13T15:42:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a335d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c06ac70 con 0x7fb49c090a40 2024-09-13T15:42:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a10f9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb49c090a40 0x7fb49c08f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a08f8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49c1261c0 0x7fb49c124140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a08f8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49c1261c0 0x7fb49c124140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43714/0 (socket says 172.21.15.70:43714) 2024-09-13T15:42:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a10f9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb49c090a40 0x7fb49c08f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41430/0 (socket says 172.21.15.70:41430) 2024-09-13T15:42:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a08f8640 1 -- 172.21.15.70:0/2950499617 learned_addr learned my addr 172.21.15.70:0/2950499617 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a18fa640 1 --2- 172.21.15.70:0/2950499617 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb49c126af0 0x7fb49c124680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a08f8640 1 -- 172.21.15.70:0/2950499617 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb49c090a40 msgr2=0x7fb49c08f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a08f8640 1 --2- 172.21.15.70:0/2950499617 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb49c090a40 0x7fb49c08f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a08f8640 1 -- 172.21.15.70:0/2950499617 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb49c126af0 msgr2=0x7fb49c124680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a08f8640 1 --2- 172.21.15.70:0/2950499617 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb49c126af0 0x7fb49c124680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a08f8640 1 -- 172.21.15.70:0/2950499617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb494009a20 con 0x7fb49c1261c0 2024-09-13T15:42:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a10f9640 1 --2- 172.21.15.70:0/2950499617 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb49c090a40 0x7fb49c08f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a08f8640 1 --2- 172.21.15.70:0/2950499617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49c1261c0 0x7fb49c124140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb49002a5b0 tx=0x7fb490032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4997fa640 1 -- 172.21.15.70:0/2950499617 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49003d720 con 0x7fb49c1261c0 2024-09-13T15:42:29.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4997fa640 1 -- 172.21.15.70:0/2950499617 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49003d880 con 0x7fb49c1261c0 2024-09-13T15:42:29.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a335d640 1 -- 172.21.15.70:0/2950499617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb490009390 con 0x7fb49c1261c0 2024-09-13T15:42:29.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4997fa640 1 -- 172.21.15.70:0/2950499617 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49003dc20 con 0x7fb49c1261c0 2024-09-13T15:42:29.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.828+0000 7fb4a335d640 1 -- 172.21.15.70:0/2950499617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49c08fb90 con 0x7fb49c1261c0 2024-09-13T15:42:29.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.829+0000 7fb4997fa640 1 -- 172.21.15.70:0/2950499617 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fb490039040 con 0x7fb49c1261c0 2024-09-13T15:42:29.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.829+0000 7fb4a335d640 1 -- 172.21.15.70:0/2950499617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49c08fe40 con 0x7fb49c1261c0 2024-09-13T15:42:29.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.830+0000 7fb4997fa640 1 -- 172.21.15.70:0/2950499617 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fb490082d60 con 0x7fb49c1261c0 2024-09-13T15:42:29.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.833+0000 7fb4997fa640 1 -- 172.21.15.70:0/2950499617 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fb49005dea0 con 0x7fb49c1261c0 2024-09-13T15:42:29.991 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.990+0000 7fb4a335d640 1 -- 172.21.15.70:0/2950499617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb49c05ed70 con 0x7fb49c1261c0 2024-09-13T15:42:29.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.992+0000 7fb4997fa640 1 -- 172.21.15.70:0/2950499617 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+116180 (secure 0 0 0) 0x7fb490042090 con 0x7fb49c1261c0 2024-09-13T15:42:29.996 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:42:29.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.994+0000 7fb4a335d640 1 -- 172.21.15.70:0/2950499617 >> v1:172.21.15.70:6816/32242 conn(0x7fb48804cb70 legacy=0x7fb48804f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:29.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.994+0000 7fb4a335d640 1 -- 172.21.15.70:0/2950499617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49c1261c0 msgr2=0x7fb49c124140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:29.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.994+0000 7fb4a335d640 1 --2- 172.21.15.70:0/2950499617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49c1261c0 0x7fb49c124140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb49002a5b0 tx=0x7fb490032000 comp rx=0 tx=0).stop 2024-09-13T15:42:29.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.994+0000 7fb4a335d640 1 -- 172.21.15.70:0/2950499617 shutdown_connections 2024-09-13T15:42:29.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.994+0000 7fb4a335d640 1 --2- 172.21.15.70:0/2950499617 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb49c126af0 0x7fb49c124680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:29.999 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.994+0000 7fb4a335d640 1 --2- 172.21.15.70:0/2950499617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb49c1261c0 0x7fb49c124140 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:29.999 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.994+0000 7fb4a335d640 1 --2- 172.21.15.70:0/2950499617 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb49c090a40 0x7fb49c08f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:29.999 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.994+0000 7fb4a335d640 1 -- 172.21.15.70:0/2950499617 >> 172.21.15.70:0/2950499617 conn(0x7fb49c0959f0 msgr2=0x7fb49c06de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:29.999 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.994+0000 7fb4a335d640 1 -- 172.21.15.70:0/2950499617 shutdown_connections 2024-09-13T15:42:29.999 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:29.994+0000 7fb4a335d640 1 -- 172.21.15.70:0/2950499617 wait complete. 2024-09-13T15:42:30.005 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6816","nonce":32242}]},"active_addr":"172.21.15.70:6816/32242","active_change":"2024-09-13T15:42:12.965910+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":3873928986}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":2914106548}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":3345804336}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":3671770748}]}]} 2024-09-13T15:42:30.008 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T15:42:30.008 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T15:42:30.008 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T15:42:30.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0bf1880640 1 Processor -- start 2024-09-13T15:42:30.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0bf1880640 1 -- start start 2024-09-13T15:42:30.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0bf1880640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0bec129f10 0x7f0bec12a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0bf1880640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0bec12a820 0x7f0bec0969d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0bf1880640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0bec096f10 0x7f0bec0973b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0bf1880640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bec098950 con 0x7f0bec096f10 2024-09-13T15:42:30.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0bf1880640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0990e0 con 0x7f0bec129f10 2024-09-13T15:42:30.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0bf1880640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec09a0d0 con 0x7f0bec12a820 2024-09-13T15:42:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0beb7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0bec096f10 0x7f0bec0973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0beaffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0bec129f10 0x7f0bec12a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0be3fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0bec12a820 0x7f0bec0969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0beb7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0bec096f10 0x7f0bec0973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43720/0 (socket says 172.21.15.70:43720) 2024-09-13T15:42:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0beaffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0bec129f10 0x7f0bec12a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:44138/0 (socket says 172.21.15.70:44138) 2024-09-13T15:42:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0beb7fe640 1 -- 172.21.15.70:0/2735676577 learned_addr learned my addr 172.21.15.70:0/2735676577 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0be3fff640 1 -- 172.21.15.70:0/2735676577 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0bec129f10 msgr2=0x7f0bec12a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0be3fff640 1 --2- 172.21.15.70:0/2735676577 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0bec129f10 0x7f0bec12a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0be3fff640 1 -- 172.21.15.70:0/2735676577 >> v2:172.21.15.70:3300/0 conn(0x7f0bec096f10 msgr2=0x7f0bec0973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0be3fff640 1 --2- 172.21.15.70:0/2735676577 >> v2:172.21.15.70:3300/0 conn(0x7f0bec096f10 0x7f0bec0973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.100+0000 7f0be3fff640 1 -- 172.21.15.70:0/2735676577 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec099d00 con 0x7f0bec12a820 2024-09-13T15:42:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.101+0000 7f0beaffd640 1 --2- 172.21.15.70:0/2735676577 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0bec129f10 0x7f0bec12a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.101+0000 7f0beb7fe640 1 --2- 172.21.15.70:0/2735676577 >> v2:172.21.15.70:3300/0 conn(0x7f0bec096f10 0x7f0bec0973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.101+0000 7f0be3fff640 1 --2- 172.21.15.70:0/2735676577 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0bec12a820 0x7f0bec0969d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009850 tx=0x7f0bd40090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa43d3940ce3b7a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.101+0000 7f0bea7fc640 1 -- 172.21.15.70:0/2735676577 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd402ac50 con 0x7f0bec12a820 2024-09-13T15:42:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.101+0000 7f0bea7fc640 1 -- 172.21.15.70:0/2735676577 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd402adb0 con 0x7f0bec12a820 2024-09-13T15:42:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.101+0000 7f0bea7fc640 1 -- 172.21.15.70:0/2735676577 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4032670 con 0x7f0bec12a820 2024-09-13T15:42:30.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.102+0000 7f0bf1880640 1 -- 172.21.15.70:0/2735676577 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0bec12a820 msgr2=0x7f0bec0969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.102+0000 7f0bf1880640 1 --2- 172.21.15.70:0/2735676577 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0bec12a820 0x7f0bec0969d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009850 tx=0x7f0bd40090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.102+0000 7f0bf1880640 1 -- 172.21.15.70:0/2735676577 shutdown_connections 2024-09-13T15:42:30.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.102+0000 7f0bf1880640 1 --2- 172.21.15.70:0/2735676577 >> v2:172.21.15.70:3300/0 conn(0x7f0bec096f10 0x7f0bec0973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.102+0000 7f0bf1880640 1 --2- 172.21.15.70:0/2735676577 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0bec12a820 0x7f0bec0969d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.102+0000 7f0bf1880640 1 --2- 172.21.15.70:0/2735676577 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0bec129f10 0x7f0bec12a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.102+0000 7f0bf1880640 1 -- 172.21.15.70:0/2735676577 >> 172.21.15.70:0/2735676577 conn(0x7f0bec09d8c0 msgr2=0x7f0bec09dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.102+0000 7f0bf1880640 1 -- 172.21.15.70:0/2735676577 shutdown_connections 2024-09-13T15:42:30.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.102+0000 7f0bf1880640 1 -- 172.21.15.70:0/2735676577 wait complete. 2024-09-13T15:42:30.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.103+0000 7f0bf1880640 1 Processor -- start 2024-09-13T15:42:30.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.103+0000 7f0bf1880640 1 -- start start 2024-09-13T15:42:30.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.103+0000 7f0bf1880640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bec096f10 0x7f0bec139540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.103+0000 7f0bf1880640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0bec129f10 0x7f0bec11ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.103+0000 7f0bf1880640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0bec12a820 0x7f0bec139b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.103+0000 7f0bf1880640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec11cc80 con 0x7f0bec096f10 2024-09-13T15:42:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.103+0000 7f0bf1880640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec11c970 con 0x7f0bec129f10 2024-09-13T15:42:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.103+0000 7f0bf1880640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec11d0e0 con 0x7f0bec12a820 2024-09-13T15:42:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.103+0000 7f0beaffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bec096f10 0x7f0bec139540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.103+0000 7f0beaffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bec096f10 0x7f0bec139540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43736/0 (socket says 172.21.15.70:43736) 2024-09-13T15:42:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.103+0000 7f0beb7fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0bec12a820 0x7f0bec139b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.103+0000 7f0beaffd640 1 -- 172.21.15.70:0/110403849 learned_addr learned my addr 172.21.15.70:0/110403849 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.103+0000 7f0be3fff640 1 --2- 172.21.15.70:0/110403849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0bec129f10 0x7f0bec11ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.104+0000 7f0beaffd640 1 -- 172.21.15.70:0/110403849 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0bec12a820 msgr2=0x7f0bec139b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.104+0000 7f0beaffd640 1 --2- 172.21.15.70:0/110403849 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0bec12a820 0x7f0bec139b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.104+0000 7f0beaffd640 1 -- 172.21.15.70:0/110403849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0bec129f10 msgr2=0x7f0bec11ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.104+0000 7f0beaffd640 1 --2- 172.21.15.70:0/110403849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0bec129f10 0x7f0bec11ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.104+0000 7f0beaffd640 1 -- 172.21.15.70:0/110403849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be4009ae0 con 0x7f0bec096f10 2024-09-13T15:42:30.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.104+0000 7f0beb7fe640 1 --2- 172.21.15.70:0/110403849 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0bec12a820 0x7f0bec139b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:30.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.104+0000 7f0beaffd640 1 --2- 172.21.15.70:0/110403849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bec096f10 0x7f0bec139540 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00ec80 tx=0x7f0bdc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.104+0000 7f0be2ffd640 1 -- 172.21.15.70:0/110403849 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc00eec0 con 0x7f0bec096f10 2024-09-13T15:42:30.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.104+0000 7f0be2ffd640 1 -- 172.21.15.70:0/110403849 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc00cbc0 con 0x7f0bec096f10 2024-09-13T15:42:30.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.104+0000 7f0bf1880640 1 -- 172.21.15.70:0/110403849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bdc009390 con 0x7f0bec096f10 2024-09-13T15:42:30.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.104+0000 7f0be2ffd640 1 -- 172.21.15.70:0/110403849 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc013470 con 0x7f0bec096f10 2024-09-13T15:42:30.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.104+0000 7f0bf1880640 1 -- 172.21.15.70:0/110403849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd4009390 con 0x7f0bec096f10 2024-09-13T15:42:30.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.105+0000 7f0be2ffd640 1 -- 172.21.15.70:0/110403849 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f0bdc0137e0 con 0x7f0bec096f10 2024-09-13T15:42:30.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.105+0000 7f0bf1880640 1 -- 172.21.15.70:0/110403849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bec05ed70 con 0x7f0bec096f10 2024-09-13T15:42:30.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.105+0000 7f0be2ffd640 1 -- 172.21.15.70:0/110403849 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f0bdc0635e0 con 0x7f0bec096f10 2024-09-13T15:42:30.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.109+0000 7f0be2ffd640 1 -- 172.21.15.70:0/110403849 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f0bdc03e550 con 0x7f0bec096f10 2024-09-13T15:42:30.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.240+0000 7f0bf1880640 1 -- 172.21.15.70:0/110403849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bec113f20 con 0x7f0bec096f10 2024-09-13T15:42:30.242 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.241+0000 7f0be2ffd640 1 -- 172.21.15.70:0/110403849 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8169 (secure 0 0 0) 0x7f0bdc041f30 con 0x7f0bec096f10 2024-09-13T15:42:30.242 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:42:30.243 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":16,"fsid":"45bbcdad-f4fb-4639-ab42-ccda8de151dd","created":"2024-09-13T15:42:09.863982+0000","modified":"2024-09-13T15:42:29.711606+0000","last_up_change":"2024-09-13T15:42:20.070768+0000","last_in_change":"2024-09-13T15:42:11.714506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T15:42:21.000499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T15:42:26.149909+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"265c6f1d-08a7-4c43-b2a6-2417ab814825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":32759}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":32759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":32759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":32759}]},"public_addr":"172.21.15.70:6804/32759","cluster_addr":"172.21.15.70:6805/32759","heartbeat_back_addr":"172.21.15.70:6807/32759","heartbeat_front_addr":"172.21.15.70:6806/32759","state":["exists","up"]},{"osd":1,"uuid":"ba4d90f9-6972-45df-800f-92ad808b06ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":32758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":32758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":32758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":32758}]},"public_addr":"172.21.15.70:6800/32758","cluster_addr":"172.21.15.70:6801/32758","heartbeat_back_addr":"172.21.15.70:6803/32758","heartbeat_front_addr":"172.21.15.70:6802/32758","state":["exists","up"]},{"osd":2,"uuid":"b74d8eb5-3082-4d69-a755-7f5f92dec5b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":32760}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":32760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":32760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":32760}]},"public_addr":"172.21.15.70:6812/32760","cluster_addr":"172.21.15.70:6813/32760","heartbeat_back_addr":"172.21.15.70:6815/32760","heartbeat_front_addr":"172.21.15.70:6814/32760","state":["exists","up"]},{"osd":3,"uuid":"02835b7d-e2cf-4ef8-87ba-b225593689d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":32769}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":32769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":32769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":32769}]},"public_addr":"172.21.15.70:6808/32769","cluster_addr":"172.21.15.70:6809/32769","heartbeat_back_addr":"172.21.15.70:6811/32769","heartbeat_front_addr":"172.21.15.70:6810/32769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.853420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.807491+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.925567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.883852+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T15:42:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.243+0000 7f0bf1880640 1 -- 172.21.15.70:0/110403849 >> v1:172.21.15.70:6816/32242 conn(0x7f0bcc04cb70 legacy=0x7f0bcc04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.243+0000 7f0bf1880640 1 -- 172.21.15.70:0/110403849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bec096f10 msgr2=0x7f0bec139540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.243+0000 7f0bf1880640 1 --2- 172.21.15.70:0/110403849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bec096f10 0x7f0bec139540 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00ec80 tx=0x7f0bdc0090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.243+0000 7f0bf1880640 1 -- 172.21.15.70:0/110403849 shutdown_connections 2024-09-13T15:42:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.243+0000 7f0bf1880640 1 --2- 172.21.15.70:0/110403849 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0bec12a820 0x7f0bec139b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.243+0000 7f0bf1880640 1 --2- 172.21.15.70:0/110403849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0bec129f10 0x7f0bec11ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.243+0000 7f0bf1880640 1 --2- 172.21.15.70:0/110403849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bec096f10 0x7f0bec139540 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.243+0000 7f0bf1880640 1 -- 172.21.15.70:0/110403849 >> 172.21.15.70:0/110403849 conn(0x7f0bec09d8c0 msgr2=0x7f0bec09c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.243+0000 7f0bf1880640 1 -- 172.21.15.70:0/110403849 shutdown_connections 2024-09-13T15:42:30.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.243+0000 7f0bf1880640 1 -- 172.21.15.70:0/110403849 wait complete. 2024-09-13T15:42:30.254 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T15:42:30.255 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T15:42:30.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.346+0000 7f4bb2d1d640 1 Processor -- start 2024-09-13T15:42:30.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.346+0000 7f4bb2d1d640 1 -- start start 2024-09-13T15:42:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4bb2d1d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4bac068110 0x7f4bac0684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4bb2d1d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bac068a20 0x7f4bac12e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4bb2d1d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4bac12e870 0x7f4bac130c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4bb2d1d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bac06a970 con 0x7f4bac12e870 2024-09-13T15:42:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4bb2d1d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac097590 con 0x7f4bac068a20 2024-09-13T15:42:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4bb2d1d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac069610 con 0x7f4bac068110 2024-09-13T15:42:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4bb12ba640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4bac12e870 0x7f4bac130c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4babfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bac068a20 0x7f4bac12e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4bb0ab9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4bac068110 0x7f4bac0684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4bb12ba640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4bac12e870 0x7f4bac130c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43750/0 (socket says 172.21.15.70:43750) 2024-09-13T15:42:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4babfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bac068a20 0x7f4bac12e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:44150/0 (socket says 172.21.15.70:44150) 2024-09-13T15:42:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4bb12ba640 1 -- 172.21.15.70:0/2459652852 learned_addr learned my addr 172.21.15.70:0/2459652852 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4bb12ba640 1 -- 172.21.15.70:0/2459652852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4bac068110 msgr2=0x7f4bac0684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4bb12ba640 1 --2- 172.21.15.70:0/2459652852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4bac068110 0x7f4bac0684e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4bb12ba640 1 -- 172.21.15.70:0/2459652852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bac068a20 msgr2=0x7f4bac12e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4bb12ba640 1 --2- 172.21.15.70:0/2459652852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bac068a20 0x7f4bac12e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4bb12ba640 1 -- 172.21.15.70:0/2459652852 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac06a130 con 0x7f4bac12e870 2024-09-13T15:42:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.347+0000 7f4babfff640 1 --2- 172.21.15.70:0/2459652852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bac068a20 0x7f4bac12e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:30.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.348+0000 7f4bb12ba640 1 --2- 172.21.15.70:0/2459652852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bac12e870 0x7f4bac130c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009f80 tx=0x7f4ba4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62308608369e8346 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.348+0000 7f4bab7fe640 1 -- 172.21.15.70:0/2459652852 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4031400 con 0x7f4bac12e870 2024-09-13T15:42:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.348+0000 7f4bab7fe640 1 -- 172.21.15.70:0/2459652852 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba402d050 con 0x7f4bac12e870 2024-09-13T15:42:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.348+0000 7f4bab7fe640 1 -- 172.21.15.70:0/2459652852 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4031790 con 0x7f4bac12e870 2024-09-13T15:42:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.348+0000 7f4bb2d1d640 1 -- 172.21.15.70:0/2459652852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bac12e870 msgr2=0x7f4bac130c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.348+0000 7f4bb2d1d640 1 --2- 172.21.15.70:0/2459652852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bac12e870 0x7f4bac130c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009f80 tx=0x7f4ba4009840 comp rx=0 tx=0).stop 2024-09-13T15:42:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.349+0000 7f4bb2d1d640 1 -- 172.21.15.70:0/2459652852 shutdown_connections 2024-09-13T15:42:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.349+0000 7f4bb2d1d640 1 --2- 172.21.15.70:0/2459652852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bac12e870 0x7f4bac130c50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.349+0000 7f4bb2d1d640 1 --2- 172.21.15.70:0/2459652852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bac068a20 0x7f4bac12e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.349+0000 7f4bb2d1d640 1 --2- 172.21.15.70:0/2459652852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4bac068110 0x7f4bac0684e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.349+0000 7f4bb2d1d640 1 -- 172.21.15.70:0/2459652852 >> 172.21.15.70:0/2459652852 conn(0x7f4bac097c90 msgr2=0x7f4bac098090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.349+0000 7f4bb2d1d640 1 -- 172.21.15.70:0/2459652852 shutdown_connections 2024-09-13T15:42:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.349+0000 7f4bb2d1d640 1 -- 172.21.15.70:0/2459652852 wait complete. 2024-09-13T15:42:30.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.349+0000 7f4bb2d1d640 1 Processor -- start 2024-09-13T15:42:30.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.349+0000 7f4bb2d1d640 1 -- start start 2024-09-13T15:42:30.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4bb2d1d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4bac068110 0x7f4bac138ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4bb2d1d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bac068a20 0x7f4bac139400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4bb2d1d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bac12e870 0x7f4bac139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4bb2d1d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac090860 con 0x7f4bac12e870 2024-09-13T15:42:30.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4bb2d1d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac121bf0 con 0x7f4bac068a20 2024-09-13T15:42:30.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4bb2d1d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac121040 con 0x7f4bac068110 2024-09-13T15:42:30.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4babfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bac068a20 0x7f4bac139400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4bb12ba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bac12e870 0x7f4bac139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4babfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bac068a20 0x7f4bac139400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:44162/0 (socket says 172.21.15.70:44162) 2024-09-13T15:42:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4bb12ba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bac12e870 0x7f4bac139940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43766/0 (socket says 172.21.15.70:43766) 2024-09-13T15:42:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4babfff640 1 -- 172.21.15.70:0/2603140452 learned_addr learned my addr 172.21.15.70:0/2603140452 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4bb0ab9640 1 --2- 172.21.15.70:0/2603140452 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4bac068110 0x7f4bac138ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4babfff640 1 -- 172.21.15.70:0/2603140452 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4bac068110 msgr2=0x7f4bac138ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4babfff640 1 --2- 172.21.15.70:0/2603140452 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4bac068110 0x7f4bac138ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4babfff640 1 -- 172.21.15.70:0/2603140452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bac12e870 msgr2=0x7f4bac139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4babfff640 1 --2- 172.21.15.70:0/2603140452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bac12e870 0x7f4bac139940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4babfff640 1 -- 172.21.15.70:0/2603140452 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba0009390 con 0x7f4bac068a20 2024-09-13T15:42:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4bb0ab9640 1 --2- 172.21.15.70:0/2603140452 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4bac068110 0x7f4bac138ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4bb12ba640 1 --2- 172.21.15.70:0/2603140452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bac12e870 0x7f4bac139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.350+0000 7f4babfff640 1 --2- 172.21.15.70:0/2603140452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bac068a20 0x7f4bac139400 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000ec50 tx=0x7f4ba00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.351+0000 7f4ba8ff9640 1 -- 172.21.15.70:0/2603140452 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba000ee90 con 0x7f4bac068a20 2024-09-13T15:42:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.351+0000 7f4ba8ff9640 1 -- 172.21.15.70:0/2603140452 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba000cbc0 con 0x7f4bac068a20 2024-09-13T15:42:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.351+0000 7f4bb2d1d640 1 -- 172.21.15.70:0/2603140452 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba4009ae0 con 0x7f4bac068a20 2024-09-13T15:42:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.351+0000 7f4ba8ff9640 1 -- 172.21.15.70:0/2603140452 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0013470 con 0x7f4bac068a20 2024-09-13T15:42:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.351+0000 7f4bb2d1d640 1 -- 172.21.15.70:0/2603140452 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bac08f620 con 0x7f4bac068a20 2024-09-13T15:42:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.352+0000 7f4bb2d1d640 1 -- 172.21.15.70:0/2603140452 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bac05ed70 con 0x7f4bac068a20 2024-09-13T15:42:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.352+0000 7f4ba8ff9640 1 -- 172.21.15.70:0/2603140452 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f4ba00137e0 con 0x7f4bac068a20 2024-09-13T15:42:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.352+0000 7f4ba8ff9640 1 -- 172.21.15.70:0/2603140452 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f4ba0063610 con 0x7f4bac068a20 2024-09-13T15:42:30.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.355+0000 7f4ba8ff9640 1 -- 172.21.15.70:0/2603140452 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f4ba003e580 con 0x7f4bac068a20 2024-09-13T15:42:30.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.486+0000 7f4bb2d1d640 1 -- 172.21.15.70:0/2603140452 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bac070480 con 0x7f4bac068a20 2024-09-13T15:42:30.488 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.487+0000 7f4ba8ff9640 1 -- 172.21.15.70:0/2603140452 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+8169 (secure 0 0 0) 0x7f4ba0041f60 con 0x7f4bac068a20 2024-09-13T15:42:30.488 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:42:30.488 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":16,"fsid":"45bbcdad-f4fb-4639-ab42-ccda8de151dd","created":"2024-09-13T15:42:09.863982+0000","modified":"2024-09-13T15:42:29.711606+0000","last_up_change":"2024-09-13T15:42:20.070768+0000","last_in_change":"2024-09-13T15:42:11.714506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T15:42:21.000499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T15:42:26.149909+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"265c6f1d-08a7-4c43-b2a6-2417ab814825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":32759}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":32759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":32759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":32759}]},"public_addr":"172.21.15.70:6804/32759","cluster_addr":"172.21.15.70:6805/32759","heartbeat_back_addr":"172.21.15.70:6807/32759","heartbeat_front_addr":"172.21.15.70:6806/32759","state":["exists","up"]},{"osd":1,"uuid":"ba4d90f9-6972-45df-800f-92ad808b06ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":32758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":32758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":32758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":32758}]},"public_addr":"172.21.15.70:6800/32758","cluster_addr":"172.21.15.70:6801/32758","heartbeat_back_addr":"172.21.15.70:6803/32758","heartbeat_front_addr":"172.21.15.70:6802/32758","state":["exists","up"]},{"osd":2,"uuid":"b74d8eb5-3082-4d69-a755-7f5f92dec5b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":32760}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":32760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":32760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":32760}]},"public_addr":"172.21.15.70:6812/32760","cluster_addr":"172.21.15.70:6813/32760","heartbeat_back_addr":"172.21.15.70:6815/32760","heartbeat_front_addr":"172.21.15.70:6814/32760","state":["exists","up"]},{"osd":3,"uuid":"02835b7d-e2cf-4ef8-87ba-b225593689d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":32769}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":32769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":32769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":32769}]},"public_addr":"172.21.15.70:6808/32769","cluster_addr":"172.21.15.70:6809/32769","heartbeat_back_addr":"172.21.15.70:6811/32769","heartbeat_front_addr":"172.21.15.70:6810/32769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.853420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.807491+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.925567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.883852+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T15:42:30.489 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.488+0000 7f4bb2d1d640 1 -- 172.21.15.70:0/2603140452 >> v1:172.21.15.70:6816/32242 conn(0x7f4b9804cb70 legacy=0x7f4b9804f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.488+0000 7f4bb2d1d640 1 -- 172.21.15.70:0/2603140452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bac068a20 msgr2=0x7f4bac139400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.488+0000 7f4bb2d1d640 1 --2- 172.21.15.70:0/2603140452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bac068a20 0x7f4bac139400 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000ec50 tx=0x7f4ba00090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.488+0000 7f4bb2d1d640 1 -- 172.21.15.70:0/2603140452 shutdown_connections 2024-09-13T15:42:30.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.488+0000 7f4bb2d1d640 1 --2- 172.21.15.70:0/2603140452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bac12e870 0x7f4bac139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.488+0000 7f4bb2d1d640 1 --2- 172.21.15.70:0/2603140452 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4bac068a20 0x7f4bac139400 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.488+0000 7f4bb2d1d640 1 --2- 172.21.15.70:0/2603140452 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4bac068110 0x7f4bac138ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.488+0000 7f4bb2d1d640 1 -- 172.21.15.70:0/2603140452 >> 172.21.15.70:0/2603140452 conn(0x7f4bac097c90 msgr2=0x7f4bac093c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.488+0000 7f4bb2d1d640 1 -- 172.21.15.70:0/2603140452 shutdown_connections 2024-09-13T15:42:30.490 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.488+0000 7f4bb2d1d640 1 -- 172.21.15.70:0/2603140452 wait complete. 2024-09-13T15:42:30.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T15:42:30.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T15:42:30.501 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T15:42:30.501 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T15:42:30.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.599+0000 7f31c6354640 1 Processor -- start 2024-09-13T15:42:30.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.599+0000 7f18c615d640 1 Processor -- start 2024-09-13T15:42:30.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.599+0000 7f7fcab9d640 1 Processor -- start 2024-09-13T15:42:30.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.599+0000 7fae69634640 1 Processor -- start 2024-09-13T15:42:30.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.599+0000 7f31c6354640 1 -- start start 2024-09-13T15:42:30.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.599+0000 7f18c615d640 1 -- start start 2024-09-13T15:42:30.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.599+0000 7fae69634640 1 -- start start 2024-09-13T15:42:30.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.599+0000 7f7fcab9d640 1 -- start start 2024-09-13T15:42:30.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7fae69634640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae64093a30 0x7fae64098ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f7fcab9d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7fc4094900 0x7f7fc40914f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7fae69634640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fae6412fa90 0x7fae64131e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7fae69634640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae64093e40 con 0x7fae640701d0 2024-09-13T15:42:30.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f7fcab9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7fc4091e40 0x7f7fc4127770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f7fcab9d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc4092590 con 0x7f7fc4091a30 2024-09-13T15:42:30.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f7fcab9d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc4093140 con 0x7f7fc4091e40 2024-09-13T15:42:30.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f7fcab9d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc4092f30 con 0x7f7fc4094900 2024-09-13T15:42:30.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7fae69634640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6406b330 con 0x7fae64093a30 2024-09-13T15:42:30.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7fae69634640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae6406ad10 con 0x7fae6412fa90 2024-09-13T15:42:30.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f31c6354640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f31c0094900 0x7f31c00914f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7fae627fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae64093a30 0x7fae64098ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7fae62ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fae640701d0 0x7fae640987e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38696/0 (socket says 172.21.15.70:38696) 2024-09-13T15:42:30.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7fae62ffd640 1 -- 172.21.15.70:0/1003922781 learned_addr learned my addr 172.21.15.70:0/1003922781 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:42:30.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f18c615d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f18c00979e0 0x7f18c012f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f18c615d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c012f6e0 0x7f18c0131ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f18c615d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c0099110 con 0x7f18c0129e90 2024-09-13T15:42:30.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f18c615d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c009a100 con 0x7f18c012f6e0 2024-09-13T15:42:30.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f18c615d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18c0098b50 con 0x7f18c00979e0 2024-09-13T15:42:30.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f31c6354640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31c0091a30 0x7f31c0091e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f7fc913a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7fc4091e40 0x7f7fc4127770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f7fc913a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7fc4091e40 0x7f7fc4127770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43772/0 (socket says 172.21.15.70:43772) 2024-09-13T15:42:30.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f7fc913a640 1 -- 172.21.15.70:0/4102906597 learned_addr learned my addr 172.21.15.70:0/4102906597 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f31c6354640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31c0127060 0x7f31c0127500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.613 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f31c6354640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31c0093140 con 0x7f31c0091a30 2024-09-13T15:42:30.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f7fc8939640 1 --2- 172.21.15.70:0/4102906597 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7fc4094900 0x7f7fc40914f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f31c6354640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c0092f30 con 0x7f31c0127060 2024-09-13T15:42:30.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f18bf7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f18c0129e90 0x7f18c012a260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38716/0 (socket says 172.21.15.70:38716) 2024-09-13T15:42:30.614 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f18bf7fe640 1 -- 172.21.15.70:0/1153802839 learned_addr learned my addr 172.21.15.70:0/1153802839 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:42:30.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f31c6354640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31c0095090 con 0x7f31c0094900 2024-09-13T15:42:30.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f31bffff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f31c0094900 0x7f31c00914f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f31bffff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f31c0094900 0x7f31c00914f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41494/0 (socket says 172.21.15.70:41494) 2024-09-13T15:42:30.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f31bffff640 1 -- 172.21.15.70:0/3441784367 learned_addr learned my addr 172.21.15.70:0/3441784367 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7f31c48f1640 1 --2- 172.21.15.70:0/3441784367 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31c0127060 0x7f31c0127500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.616 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.600+0000 7fae637fe640 1 --2- 172.21.15.70:0/1003922781 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fae6412fa90 0x7fae64131e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18bffff640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c012f6e0 0x7f18c0131ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18beffd640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f18c00979e0 0x7f18c012f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 -- 172.21.15.70:0/1153802839 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199415129 0 0) 0x7f18c0099110 con 0x7f18c0129e90 2024-09-13T15:42:30.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 -- 172.21.15.70:0/1153802839 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18ac0034e0 con 0x7f18c0129e90 2024-09-13T15:42:30.618 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f31bf7fe640 1 --2- 172.21.15.70:0/3441784367 >> v2:172.21.15.70:3300/0 conn(0x7f31c0091a30 0x7f31c0091e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.618 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f7fc8939640 1 -- 172.21.15.70:0/4102906597 >> v2:172.21.15.70:3300/0 conn(0x7f7fc4091e40 msgr2=0x7f7fc4127770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.618 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f7fc8939640 1 --2- 172.21.15.70:0/4102906597 >> v2:172.21.15.70:3300/0 conn(0x7f7fc4091e40 0x7f7fc4127770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.619 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f7fc8939640 1 -- 172.21.15.70:0/4102906597 >> v1:172.21.15.70:6789/0 conn(0x7f7fc4091a30 legacy=0x7f7fc4127060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.619 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f7fc8939640 1 -- 172.21.15.70:0/4102906597 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc40927c0 con 0x7f7fc4094900 2024-09-13T15:42:30.620 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f31bffff640 1 -- 172.21.15.70:0/3441784367 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31c0127060 msgr2=0x7f31c0127500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.620 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f31bffff640 1 --2- 172.21.15.70:0/3441784367 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31c0127060 0x7f31c0127500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.620 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f31bffff640 1 -- 172.21.15.70:0/3441784367 >> v2:172.21.15.70:3300/0 conn(0x7f31c0091a30 msgr2=0x7f31c0091e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.620 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f31bffff640 1 --2- 172.21.15.70:0/3441784367 >> v2:172.21.15.70:3300/0 conn(0x7f31c0091a30 0x7f31c0091e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.620 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f31bffff640 1 -- 172.21.15.70:0/3441784367 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c0092590 con 0x7f31c0094900 2024-09-13T15:42:30.622 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 -- 172.21.15.70:0/1153802839 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 714636074 0 0) 0x7f18ac0034e0 con 0x7f18c0129e90 2024-09-13T15:42:30.622 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 -- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f18c00979e0 msgr2=0x7f18c012f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.622 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f18c00979e0 0x7f18c012f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.622 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 -- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c012f6e0 msgr2=0x7f18c0131ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.622 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c012f6e0 0x7f18c0131ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.622 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 -- 172.21.15.70:0/1153802839 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c00995c0 con 0x7f18c0129e90 2024-09-13T15:42:30.622 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 -- 172.21.15.70:0/1153802839 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2737531732 0 0) 0x7f18b0001470 con 0x7f18c0129e90 2024-09-13T15:42:30.622 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 -- 172.21.15.70:0/1153802839 >> v1:172.21.15.70:6789/0 conn(0x7f18c0129e90 legacy=0x7f18c012a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.622 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f18ac003a50 0x7f18ac003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.623 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac004780 0x7f18ac004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.623 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c012f6e0 0x7f18ac005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.623 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 -- 172.21.15.70:0/1153802839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c009a100 con 0x7f18ac004780 2024-09-13T15:42:30.623 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 -- 172.21.15.70:0/1153802839 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c0098b50 con 0x7f18c012f6e0 2024-09-13T15:42:30.623 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18be7fc640 1 -- 172.21.15.70:0/1153802839 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac00bb00 con 0x7f18ac003a50 2024-09-13T15:42:30.623 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7f18bffff640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c012f6e0 0x7f18ac005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.623 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7fae627fc640 1 -- 172.21.15.70:0/1003922781 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fae6412fa90 msgr2=0x7fae64131e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7fae627fc640 1 --2- 172.21.15.70:0/1003922781 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fae6412fa90 0x7fae64131e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7fae627fc640 1 -- 172.21.15.70:0/1003922781 >> v1:172.21.15.70:6789/0 conn(0x7fae640701d0 legacy=0x7fae640987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.624 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.601+0000 7fae627fc640 1 -- 172.21.15.70:0/1003922781 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae64070860 con 0x7fae64093a30 2024-09-13T15:42:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7f18bf7fe640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f18ac003a50 0x7f18ac003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7f18beffd640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f18c00979e0 0x7f18c012f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:42:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7f18bffff640 1 -- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f18ac003a50 msgr2=0x7f18ac003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7f18bffff640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f18ac003a50 0x7f18ac003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7f18bffff640 1 -- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac004780 msgr2=0x7f18ac004bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7f18bffff640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac004780 0x7f18ac004bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7f18bffff640 1 -- 172.21.15.70:0/1153802839 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac0034e0 con 0x7f18c012f6e0 2024-09-13T15:42:30.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7fae627fc640 1 --2- 172.21.15.70:0/1003922781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae64093a30 0x7fae64098ef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fae58009880 tx=0x7fae580090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddfe3ab5438b4d2a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7fae61ffb640 1 -- 172.21.15.70:0/1003922781 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5802ace0 con 0x7fae64093a30 2024-09-13T15:42:30.626 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7f7fc8939640 1 --2- 172.21.15.70:0/4102906597 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7fc4094900 0x7f7fc40914f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009850 tx=0x7f7fb80090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a01da6cc74ef4c65 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7fae61ffb640 1 -- 172.21.15.70:0/1003922781 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5802ae40 con 0x7fae64093a30 2024-09-13T15:42:30.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7fae61ffb640 1 -- 172.21.15.70:0/1003922781 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58032670 con 0x7fae64093a30 2024-09-13T15:42:30.627 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7f31bffff640 1 --2- 172.21.15.70:0/3441784367 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f31c0094900 0x7f31c00914f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f31b000ba00 tx=0x7f31b00090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b51dcccc9a9251f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7f7fc37fe640 1 -- 172.21.15.70:0/4102906597 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb802ac80 con 0x7f7fc4094900 2024-09-13T15:42:30.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7f7fc37fe640 1 -- 172.21.15.70:0/4102906597 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb802ade0 con 0x7f7fc4094900 2024-09-13T15:42:30.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7f7fc37fe640 1 -- 172.21.15.70:0/4102906597 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8032670 con 0x7f7fc4094900 2024-09-13T15:42:30.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7f31beffd640 1 -- 172.21.15.70:0/3441784367 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b0031400 con 0x7f31c0094900 2024-09-13T15:42:30.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7f31beffd640 1 -- 172.21.15.70:0/3441784367 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b002d050 con 0x7f31c0094900 2024-09-13T15:42:30.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.602+0000 7f31beffd640 1 -- 172.21.15.70:0/3441784367 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b00317d0 con 0x7f31c0094900 2024-09-13T15:42:30.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f18bffff640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c012f6e0 0x7f18ac005130 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18b800e210 tx=0x7f18b800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92bec397893d725c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7fae69634640 1 -- 172.21.15.70:0/1003922781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae64093a30 msgr2=0x7fae64098ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7fae69634640 1 --2- 172.21.15.70:0/1003922781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae64093a30 0x7fae64098ef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fae58009880 tx=0x7fae580090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f18be7fc640 1 -- 172.21.15.70:0/1153802839 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b802c040 con 0x7f18c012f6e0 2024-09-13T15:42:30.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7fae69634640 1 -- 172.21.15.70:0/1003922781 shutdown_connections 2024-09-13T15:42:30.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7fae69634640 1 --2- 172.21.15.70:0/1003922781 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fae6412fa90 0x7fae64131e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7fae69634640 1 --2- 172.21.15.70:0/1003922781 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae64093a30 0x7fae64098ef0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.631 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7fae69634640 1 -- 172.21.15.70:0/1003922781 >> 172.21.15.70:0/1003922781 conn(0x7fae640963c0 msgr2=0x7fae640967c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f18be7fc640 1 -- 172.21.15.70:0/1153802839 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b800f040 con 0x7f18c012f6e0 2024-09-13T15:42:30.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f18be7fc640 1 -- 172.21.15.70:0/1153802839 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8030630 con 0x7f18c012f6e0 2024-09-13T15:42:30.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f7fcab9d640 1 -- 172.21.15.70:0/4102906597 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7fc4094900 msgr2=0x7f7fc40914f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f7fcab9d640 1 --2- 172.21.15.70:0/4102906597 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7fc4094900 0x7f7fc40914f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009850 tx=0x7f7fb80090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7fae69634640 1 -- 172.21.15.70:0/1003922781 shutdown_connections 2024-09-13T15:42:30.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7fae69634640 1 -- 172.21.15.70:0/1003922781 wait complete. 2024-09-13T15:42:30.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f7fcab9d640 1 -- 172.21.15.70:0/4102906597 shutdown_connections 2024-09-13T15:42:30.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f7fcab9d640 1 --2- 172.21.15.70:0/4102906597 >> v2:172.21.15.70:3300/0 conn(0x7f7fc4091e40 0x7f7fc4127770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f7fcab9d640 1 --2- 172.21.15.70:0/4102906597 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7fc4094900 0x7f7fc40914f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f7fcab9d640 1 -- 172.21.15.70:0/4102906597 >> 172.21.15.70:0/4102906597 conn(0x7f7fc40959f0 msgr2=0x7f7fc4095df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f31c6354640 1 -- 172.21.15.70:0/3441784367 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f31c0094900 msgr2=0x7f31c00914f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.634 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f31c6354640 1 --2- 172.21.15.70:0/3441784367 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f31c0094900 0x7f31c00914f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f31b000ba00 tx=0x7f31b00090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f7fcab9d640 1 -- 172.21.15.70:0/4102906597 shutdown_connections 2024-09-13T15:42:30.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f7fcab9d640 1 -- 172.21.15.70:0/4102906597 wait complete. 2024-09-13T15:42:30.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f31c6354640 1 -- 172.21.15.70:0/3441784367 shutdown_connections 2024-09-13T15:42:30.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f31c6354640 1 --2- 172.21.15.70:0/3441784367 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31c0127060 0x7f31c0127500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f31c6354640 1 --2- 172.21.15.70:0/3441784367 >> v2:172.21.15.70:3300/0 conn(0x7f31c0091a30 0x7f31c0091e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f31c6354640 1 --2- 172.21.15.70:0/3441784367 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f31c0094900 0x7f31c00914f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f31c6354640 1 -- 172.21.15.70:0/3441784367 >> 172.21.15.70:0/3441784367 conn(0x7f31c00959f0 msgr2=0x7f31c0095df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f31c6354640 1 -- 172.21.15.70:0/3441784367 shutdown_connections 2024-09-13T15:42:30.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.603+0000 7f31c6354640 1 -- 172.21.15.70:0/3441784367 wait complete. 2024-09-13T15:42:30.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.604+0000 7f18c615d640 1 -- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c012f6e0 msgr2=0x7f18ac005130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.604+0000 7f18c615d640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c012f6e0 0x7f18ac005130 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18b800e210 tx=0x7f18b800b040 comp rx=0 tx=0).stop 2024-09-13T15:42:30.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.604+0000 7f18c615d640 1 -- 172.21.15.70:0/1153802839 shutdown_connections 2024-09-13T15:42:30.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.604+0000 7f18c615d640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18c012f6e0 0x7f18ac005130 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.604+0000 7f18c615d640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac004780 0x7f18ac004bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.604+0000 7f18c615d640 1 --2- 172.21.15.70:0/1153802839 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f18ac003a50 0x7f18ac003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.604+0000 7f18c615d640 1 -- 172.21.15.70:0/1153802839 >> 172.21.15.70:0/1153802839 conn(0x7f18c009d840 msgr2=0x7f18c009dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.604+0000 7f18c615d640 1 -- 172.21.15.70:0/1153802839 shutdown_connections 2024-09-13T15:42:30.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.604+0000 7f18c615d640 1 -- 172.21.15.70:0/1153802839 wait complete. 2024-09-13T15:42:30.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.604+0000 7f31c6354640 1 Processor -- start 2024-09-13T15:42:30.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.604+0000 7fae69634640 1 Processor -- start 2024-09-13T15:42:30.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.604+0000 7fae69634640 1 -- start start 2024-09-13T15:42:30.639 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.604+0000 7f31c6354640 1 -- start start 2024-09-13T15:42:30.640 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f7fcab9d640 1 Processor -- start 2024-09-13T15:42:30.640 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f31c6354640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31c0091a30 0x7f31c01352f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f31c6354640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f31c0094900 0x7f31c0135830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f31c6354640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31c0127060 0x7f31c008c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f31c6354640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c008f920 con 0x7f31c0091a30 2024-09-13T15:42:30.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f31c6354640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c008dbd0 con 0x7f31c0127060 2024-09-13T15:42:30.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f31c6354640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31c008f660 con 0x7f31c0094900 2024-09-13T15:42:30.641 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f18c615d640 1 Processor -- start 2024-09-13T15:42:30.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f7fcab9d640 1 -- start start 2024-09-13T15:42:30.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f31bffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31c0091a30 0x7f31c01352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f31bf7fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f31c0094900 0x7f31c0135830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.642 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f31bffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31c0091a30 0x7f31c01352f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43790/0 (socket says 172.21.15.70:43790) 2024-09-13T15:42:30.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f18c615d640 1 -- start start 2024-09-13T15:42:30.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7fae69634640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fae640701d0 0x7fae6411f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7fae69634640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae64093a30 0x7fae6411f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7fae69634640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae6412fa90 0x7fae6411fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7fae69634640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae64121ec0 con 0x7fae6412fa90 2024-09-13T15:42:30.643 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7fae69634640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6412f760 con 0x7fae64093a30 2024-09-13T15:42:30.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7fae69634640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae641226a0 con 0x7fae640701d0 2024-09-13T15:42:30.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f31c48f1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31c0127060 0x7f31c008c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f31bf7fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f31c0094900 0x7f31c0135830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41528/0 (socket says 172.21.15.70:41528) 2024-09-13T15:42:30.644 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7f31bffff640 1 -- 172.21.15.70:0/1042817245 learned_addr learned my addr 172.21.15.70:0/1042817245 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7fae627fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae64093a30 0x7fae6411f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7fae627fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae64093a30 0x7fae6411f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:44212/0 (socket says 172.21.15.70:44212) 2024-09-13T15:42:30.645 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7fae627fc640 1 -- 172.21.15.70:0/4000347256 learned_addr learned my addr 172.21.15.70:0/4000347256 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7fae637fe640 1 --2- 172.21.15.70:0/4000347256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae6412fa90 0x7fae6411fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.605+0000 7fae62ffd640 1 --2- 172.21.15.70:0/4000347256 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fae640701d0 0x7fae6411f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f31bf7fe640 1 -- 172.21.15.70:0/1042817245 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31c0127060 msgr2=0x7f31c008c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.646 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f31bf7fe640 1 --2- 172.21.15.70:0/1042817245 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31c0127060 0x7f31c008c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f31bf7fe640 1 -- 172.21.15.70:0/1042817245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31c0091a30 msgr2=0x7f31c01352f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f31bf7fe640 1 --2- 172.21.15.70:0/1042817245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31c0091a30 0x7f31c01352f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f31bf7fe640 1 -- 172.21.15.70:0/1042817245 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b000b510 con 0x7f31c0094900 2024-09-13T15:42:30.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f31c48f1640 1 --2- 172.21.15.70:0/1042817245 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31c0127060 0x7f31c008c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:30.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7fae627fc640 1 -- 172.21.15.70:0/4000347256 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fae640701d0 msgr2=0x7fae6411f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7fae627fc640 1 --2- 172.21.15.70:0/4000347256 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fae640701d0 0x7fae6411f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.647 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7fae627fc640 1 -- 172.21.15.70:0/4000347256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae6412fa90 msgr2=0x7fae6411fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f31bf7fe640 1 --2- 172.21.15.70:0/1042817245 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f31c0094900 0x7f31c0135830 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f31c0099f00 tx=0x7f31b4008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18c615d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac003a50 0x7f18c0138e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18c615d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18ac004780 0x7f18c0139390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18c615d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f18c0129e90 0x7f18c0118a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18c615d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c011bf00 con 0x7f18ac003a50 2024-09-13T15:42:30.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18c615d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c011b580 con 0x7f18ac004780 2024-09-13T15:42:30.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18c615d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18c011c130 con 0x7f18c0129e90 2024-09-13T15:42:30.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18beffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18ac004780 0x7f18c0139390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18bf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac003a50 0x7f18c0138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18beffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18ac004780 0x7f18c0139390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:44226/0 (socket says 172.21.15.70:44226) 2024-09-13T15:42:30.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18bffff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f18c0129e90 0x7f18c0118a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18beffd640 1 -- 172.21.15.70:0/716539145 learned_addr learned my addr 172.21.15.70:0/716539145 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7fae627fc640 1 --2- 172.21.15.70:0/4000347256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae6412fa90 0x7fae6411fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7fae627fc640 1 -- 172.21.15.70:0/4000347256 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae58009390 con 0x7fae64093a30 2024-09-13T15:42:30.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7fae637fe640 1 --2- 172.21.15.70:0/4000347256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae6412fa90 0x7fae6411fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:30.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f7fcab9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fc4091a30 0x7f7fc4135080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f31a7fff640 1 -- 172.21.15.70:0/1042817245 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4018070 con 0x7f31c0094900 2024-09-13T15:42:30.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18beffd640 1 -- 172.21.15.70:0/716539145 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f18c0129e90 msgr2=0x7f18c0118a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f7fcab9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fc4091e40 0x7f7fc41355c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f31c6354640 1 -- 172.21.15.70:0/1042817245 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c008f220 con 0x7f31c0094900 2024-09-13T15:42:30.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f31a7fff640 1 -- 172.21.15.70:0/1042817245 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b4015ac0 con 0x7f31c0094900 2024-09-13T15:42:30.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18beffd640 1 --2- 172.21.15.70:0/716539145 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f18c0129e90 0x7f18c0118a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18beffd640 1 -- 172.21.15.70:0/716539145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac003a50 msgr2=0x7f18c0138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18beffd640 1 --2- 172.21.15.70:0/716539145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac003a50 0x7f18c0138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f18beffd640 1 -- 172.21.15.70:0/716539145 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c01398d0 con 0x7f18ac004780 2024-09-13T15:42:30.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f31a7fff640 1 -- 172.21.15.70:0/1042817245 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4018070 con 0x7f31c0094900 2024-09-13T15:42:30.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f31c6354640 1 -- 172.21.15.70:0/1042817245 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c008ec60 con 0x7f31c0094900 2024-09-13T15:42:30.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f7fcab9d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7fc4094900 0x7f7fc4135b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7fae627fc640 1 --2- 172.21.15.70:0/4000347256 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae64093a30 0x7fae6411f7d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fae58009360 tx=0x7fae58032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f7fcab9d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc406df10 con 0x7f7fc4091e40 2024-09-13T15:42:30.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f7fcab9d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc408e760 con 0x7f7fc4091a30 2024-09-13T15:42:30.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f7fcab9d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc411e380 con 0x7f7fc4094900 2024-09-13T15:42:30.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f18bf7fe640 1 --2- 172.21.15.70:0/716539145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac003a50 0x7f18c0138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:42:30.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f7fc3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fc4091e40 0x7f7fc41355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f7fc3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fc4091e40 0x7f7fc41355c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43820/0 (socket says 172.21.15.70:43820) 2024-09-13T15:42:30.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f18beffd640 1 --2- 172.21.15.70:0/716539145 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18ac004780 0x7f18c0139390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f18a800d840 tx=0x7f18a8008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7fae4affd640 1 -- 172.21.15.70:0/4000347256 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58031700 con 0x7fae64093a30 2024-09-13T15:42:30.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7fae4affd640 1 -- 172.21.15.70:0/4000347256 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae58031860 con 0x7fae64093a30 2024-09-13T15:42:30.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7fae69634640 1 -- 172.21.15.70:0/4000347256 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae64122a70 con 0x7fae64093a30 2024-09-13T15:42:30.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7fae69634640 1 -- 172.21.15.70:0/4000347256 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae64120470 con 0x7fae64093a30 2024-09-13T15:42:30.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7fae4affd640 1 -- 172.21.15.70:0/4000347256 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58031c00 con 0x7fae64093a30 2024-09-13T15:42:30.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.606+0000 7f7fc3fff640 1 -- 172.21.15.70:0/204115067 learned_addr learned my addr 172.21.15.70:0/204115067 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f7fc913a640 1 --2- 172.21.15.70:0/204115067 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7fc4094900 0x7f7fc4135b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.658 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f7fc3fff640 1 -- 172.21.15.70:0/204115067 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7fc4094900 msgr2=0x7f7fc4135b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f7fc8939640 1 --2- 172.21.15.70:0/204115067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fc4091a30 0x7f7fc4135080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f7fc3fff640 1 --2- 172.21.15.70:0/204115067 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7fc4094900 0x7f7fc4135b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f7fc3fff640 1 -- 172.21.15.70:0/204115067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fc4091a30 msgr2=0x7f7fc4135080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f7fc3fff640 1 --2- 172.21.15.70:0/204115067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fc4091a30 0x7f7fc4135080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f7fc3fff640 1 -- 172.21.15.70:0/204115067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb8009390 con 0x7f7fc4091e40 2024-09-13T15:42:30.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f7fc8939640 1 --2- 172.21.15.70:0/204115067 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7fc4091a30 0x7f7fc4135080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:42:30.659 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f7fc3fff640 1 --2- 172.21.15.70:0/204115067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fc4091e40 0x7f7fc41355c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00017c0 tx=0x7f7fb0007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f18b77fe640 1 -- 172.21.15.70:0/716539145 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a80125f0 con 0x7f18ac004780 2024-09-13T15:42:30.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f18c615d640 1 -- 172.21.15.70:0/716539145 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c011ad80 con 0x7f18ac004780 2024-09-13T15:42:30.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f18c615d640 1 -- 172.21.15.70:0/716539145 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c0119110 con 0x7f18ac004780 2024-09-13T15:42:30.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f18b77fe640 1 -- 172.21.15.70:0/716539145 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a8012770 con 0x7f18ac004780 2024-09-13T15:42:30.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.607+0000 7f18b77fe640 1 -- 172.21.15.70:0/716539145 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8012b30 con 0x7f18ac004780 2024-09-13T15:42:30.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.608+0000 7f7fc913a640 1 --2- 172.21.15.70:0/204115067 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7fc4094900 0x7f7fc4135b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:42:30.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.608+0000 7f7fc0ff9640 1 -- 172.21.15.70:0/204115067 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb000a7f0 con 0x7f7fc4091e40 2024-09-13T15:42:30.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.608+0000 7f7fc0ff9640 1 -- 172.21.15.70:0/204115067 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb000a970 con 0x7f7fc4091e40 2024-09-13T15:42:30.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.608+0000 7f7fc0ff9640 1 -- 172.21.15.70:0/204115067 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb000ad30 con 0x7f7fc4091e40 2024-09-13T15:42:30.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.608+0000 7f7fcab9d640 1 -- 172.21.15.70:0/204115067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc411dfd0 con 0x7f7fc4091e40 2024-09-13T15:42:30.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.609+0000 7fae4affd640 1 -- 172.21.15.70:0/4000347256 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fae58039040 con 0x7fae64093a30 2024-09-13T15:42:30.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.608+0000 7f7fcab9d640 1 -- 172.21.15.70:0/204115067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc408ce30 con 0x7f7fc4091e40 2024-09-13T15:42:30.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.609+0000 7f18c615d640 1 -- 172.21.15.70:0/716539145 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f18a4000fd0 con 0x7f18ac004780 2024-09-13T15:42:30.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.608+0000 7f31c6354640 1 -- 172.21.15.70:0/1042817245 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f31c005ed70 con 0x7f31c0094900 2024-09-13T15:42:30.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.609+0000 7f31a7fff640 1 -- 172.21.15.70:0/1042817245 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f31b401e400 con 0x7f31c0094900 2024-09-13T15:42:30.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.609+0000 7fae69634640 1 -- 172.21.15.70:0/4000347256 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fae6405ed70 con 0x7fae64093a30 2024-09-13T15:42:30.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.609+0000 7f31a7fff640 1 -- 172.21.15.70:0/1042817245 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f31b406ac00 con 0x7f31c0094900 2024-09-13T15:42:30.664 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.609+0000 7f7fae7fc640 1 -- 172.21.15.70:0/204115067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7fc405ed70 con 0x7f7fc4091e40 2024-09-13T15:42:30.664 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.609+0000 7fae4affd640 1 -- 172.21.15.70:0/4000347256 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fae58083510 con 0x7fae64093a30 2024-09-13T15:42:30.664 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.609+0000 7fae4affd640 1 -- 172.21.15.70:0/4000347256 --> v1:172.21.15.70:6800/32758 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae50056ed0 con 0x7fae500531f0 2024-09-13T15:42:30.664 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.609+0000 7fae4affd640 1 -- 172.21.15.70:0/4000347256 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae58062460 con 0x7fae64093a30 2024-09-13T15:42:30.665 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.609+0000 7f31a7fff640 1 -- 172.21.15.70:0/1042817245 --> v1:172.21.15.70:6812/32760 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31ac056f30 con 0x7f31ac053250 2024-09-13T15:42:30.665 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.609+0000 7f31a7fff640 1 -- 172.21.15.70:0/1042817245 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31b406d220 con 0x7f31c0094900 2024-09-13T15:42:30.665 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.610+0000 7f7fc0ff9640 1 -- 172.21.15.70:0/204115067 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f7fb0018610 con 0x7f7fc4091e40 2024-09-13T15:42:30.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.610+0000 7f18b77fe640 1 -- 172.21.15.70:0/716539145 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f18a8013480 con 0x7f18ac004780 2024-09-13T15:42:30.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.610+0000 7f7fc0ff9640 1 -- 172.21.15.70:0/204115067 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f7fb0066aa0 con 0x7f7fc4091e40 2024-09-13T15:42:30.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.610+0000 7f7fc0ff9640 1 -- 172.21.15.70:0/204115067 --> v1:172.21.15.70:6804/32759 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fa8057020 con 0x7f7fa8053340 2024-09-13T15:42:30.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.610+0000 7f7fc0ff9640 1 -- 172.21.15.70:0/204115067 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fb006b220 con 0x7f7fc4091e40 2024-09-13T15:42:30.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.610+0000 7f18b77fe640 1 -- 172.21.15.70:0/716539145 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f18a8063130 con 0x7f18ac004780 2024-09-13T15:42:30.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.610+0000 7f18b77fe640 1 -- 172.21.15.70:0/716539145 --> v1:172.21.15.70:6808/32769 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18a0056fa0 con 0x7f18a00532c0 2024-09-13T15:42:30.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.610+0000 7f18b77fe640 1 -- 172.21.15.70:0/716539145 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18a8065750 con 0x7f18ac004780 2024-09-13T15:42:30.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.611+0000 7f31a7fff640 1 -- 172.21.15.70:0/1042817245 <== osd.2 v1:172.21.15.70:6812/32760 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f31ac056f30 con 0x7f31ac053250 2024-09-13T15:42:30.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.611+0000 7fae4affd640 1 -- 172.21.15.70:0/4000347256 <== osd.1 v1:172.21.15.70:6800/32758 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fae50056ed0 con 0x7fae500531f0 2024-09-13T15:42:30.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.611+0000 7f18b77fe640 1 -- 172.21.15.70:0/716539145 <== osd.3 v1:172.21.15.70:6808/32769 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f18a0056fa0 con 0x7f18a00532c0 2024-09-13T15:42:30.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.611+0000 7f7fc0ff9640 1 -- 172.21.15.70:0/204115067 <== osd.0 v1:172.21.15.70:6804/32759 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f7fa8057020 con 0x7f7fa8053340 2024-09-13T15:42:30.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.623+0000 7fae69634640 1 -- 172.21.15.70:0/4000347256 --> v1:172.21.15.70:6800/32758 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fae6406f140 con 0x7fae500531f0 2024-09-13T15:42:30.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.623+0000 7f31c6354640 1 -- 172.21.15.70:0/1042817245 --> v1:172.21.15.70:6812/32760 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f31c008c900 con 0x7f31ac053250 2024-09-13T15:42:30.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.623+0000 7fae4affd640 1 -- 172.21.15.70:0/4000347256 <== osd.1 v1:172.21.15.70:6800/32758 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2217796425) 0x7fae5c0074f0 con 0x7fae500531f0 2024-09-13T15:42:30.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.623+0000 7f31a7fff640 1 -- 172.21.15.70:0/1042817245 <== osd.2 v1:172.21.15.70:6812/32760 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3659784045) 0x7f31c008c900 con 0x7f31ac053250 2024-09-13T15:42:30.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7f7fae7fc640 1 -- 172.21.15.70:0/204115067 --> v1:172.21.15.70:6804/32759 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7fc408d370 con 0x7f7fa8053340 2024-09-13T15:42:30.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7f18c615d640 1 -- 172.21.15.70:0/716539145 --> v1:172.21.15.70:6808/32769 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f18a4002c70 con 0x7f18a00532c0 2024-09-13T15:42:30.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7f31c6354640 1 -- 172.21.15.70:0/1042817245 >> v1:172.21.15.70:6812/32760 conn(0x7f31ac053250 legacy=0x7f31ac0556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7fae69634640 1 -- 172.21.15.70:0/4000347256 >> v1:172.21.15.70:6800/32758 conn(0x7fae500531f0 legacy=0x7fae50055660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7f31c6354640 1 -- 172.21.15.70:0/1042817245 >> v1:172.21.15.70:6816/32242 conn(0x7f31ac04cbd0 legacy=0x7f31ac04f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7f31c6354640 1 -- 172.21.15.70:0/1042817245 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f31c0094900 msgr2=0x7f31c0135830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7f31c6354640 1 --2- 172.21.15.70:0/1042817245 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f31c0094900 0x7f31c0135830 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f31c0099f00 tx=0x7f31b4008040 comp rx=0 tx=0).stop 2024-09-13T15:42:30.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7f31c48f1640 1 -- 172.21.15.70:0/1042817245 reap_dead start 2024-09-13T15:42:30.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7f31c6354640 1 -- 172.21.15.70:0/1042817245 shutdown_connections 2024-09-13T15:42:30.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7f31c6354640 1 -- 172.21.15.70:0/1042817245 >> 172.21.15.70:0/1042817245 conn(0x7f31c00959f0 msgr2=0x7f31c006b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7f18b77fe640 1 -- 172.21.15.70:0/716539145 <== osd.3 v1:172.21.15.70:6808/32769 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3659784045) 0x7f18a4002c70 con 0x7f18a00532c0 2024-09-13T15:42:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7fae69634640 1 -- 172.21.15.70:0/4000347256 >> v1:172.21.15.70:6816/32242 conn(0x7fae5004cb70 legacy=0x7fae5004f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7fae69634640 1 -- 172.21.15.70:0/4000347256 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae64093a30 msgr2=0x7fae6411f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7fae69634640 1 --2- 172.21.15.70:0/4000347256 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae64093a30 0x7fae6411f7d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fae58009360 tx=0x7fae58032000 comp rx=0 tx=0).stop 2024-09-13T15:42:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7fae637fe640 1 -- 172.21.15.70:0/4000347256 reap_dead start 2024-09-13T15:42:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7fae69634640 1 -- 172.21.15.70:0/4000347256 shutdown_connections 2024-09-13T15:42:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7fae69634640 1 -- 172.21.15.70:0/4000347256 >> 172.21.15.70:0/4000347256 conn(0x7fae640963c0 msgr2=0x7fae64094e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7fae69634640 1 -- 172.21.15.70:0/4000347256 shutdown_connections 2024-09-13T15:42:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7fae69634640 1 -- 172.21.15.70:0/4000347256 wait complete. 2024-09-13T15:42:30.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7f7fc0ff9640 1 -- 172.21.15.70:0/204115067 <== osd.0 v1:172.21.15.70:6804/32759 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3659784045) 0x7f7fc408d370 con 0x7f7fa8053340 2024-09-13T15:42:30.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7f31c6354640 1 -- 172.21.15.70:0/1042817245 shutdown_connections 2024-09-13T15:42:30.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.624+0000 7f31c6354640 1 -- 172.21.15.70:0/1042817245 wait complete. 2024-09-13T15:42:30.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f18c615d640 1 -- 172.21.15.70:0/716539145 >> v1:172.21.15.70:6808/32769 conn(0x7f18a00532c0 legacy=0x7f18a0055730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f7fae7fc640 1 -- 172.21.15.70:0/204115067 >> v1:172.21.15.70:6804/32759 conn(0x7f7fa8053340 legacy=0x7f7fa80557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.677 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f18c615d640 1 -- 172.21.15.70:0/716539145 >> v1:172.21.15.70:6816/32242 conn(0x7f18a004cc40 legacy=0x7f18a004f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f7fae7fc640 1 -- 172.21.15.70:0/204115067 >> v1:172.21.15.70:6816/32242 conn(0x7f7fa804ccc0 legacy=0x7f7fa804f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f18c615d640 1 -- 172.21.15.70:0/716539145 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18ac004780 msgr2=0x7f18c0139390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f18c615d640 1 --2- 172.21.15.70:0/716539145 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f18ac004780 0x7f18c0139390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f18a800d840 tx=0x7f18a8008040 comp rx=0 tx=0).stop 2024-09-13T15:42:30.678 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f7fae7fc640 1 -- 172.21.15.70:0/204115067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fc4091e40 msgr2=0x7f7fc41355c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f7fae7fc640 1 --2- 172.21.15.70:0/204115067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fc4091e40 0x7f7fc41355c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00017c0 tx=0x7f7fb0007ec0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f7fc913a640 1 -- 172.21.15.70:0/204115067 reap_dead start 2024-09-13T15:42:30.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f18bffff640 1 -- 172.21.15.70:0/716539145 reap_dead start 2024-09-13T15:42:30.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f18c615d640 1 -- 172.21.15.70:0/716539145 shutdown_connections 2024-09-13T15:42:30.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f18c615d640 1 -- 172.21.15.70:0/716539145 >> 172.21.15.70:0/716539145 conn(0x7f18c009d840 msgr2=0x7f18c009b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f7fae7fc640 1 -- 172.21.15.70:0/204115067 shutdown_connections 2024-09-13T15:42:30.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f7fae7fc640 1 -- 172.21.15.70:0/204115067 >> 172.21.15.70:0/204115067 conn(0x7f7fc40959f0 msgr2=0x7f7fc406bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f18c615d640 1 -- 172.21.15.70:0/716539145 shutdown_connections 2024-09-13T15:42:30.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f7fae7fc640 1 -- 172.21.15.70:0/204115067 shutdown_connections 2024-09-13T15:42:30.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f7fae7fc640 1 -- 172.21.15.70:0/204115067 wait complete. 2024-09-13T15:42:30.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.625+0000 7f18c615d640 1 -- 172.21.15.70:0/716539145 wait complete. 2024-09-13T15:42:30.681 INFO:teuthology.orchestra.run.smithi070.stdout:38654705668 2024-09-13T15:42:30.682 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T15:42:30.683 INFO:teuthology.orchestra.run.smithi070.stdout:38654705668 2024-09-13T15:42:30.683 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T15:42:30.683 INFO:teuthology.orchestra.run.smithi070.stdout:38654705667 2024-09-13T15:42:30.684 INFO:teuthology.orchestra.run.smithi070.stdout:38654705668 2024-09-13T15:42:30.684 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T15:42:30.685 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T15:42:30.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.819+0000 7fc4dbc78640 1 Processor -- start 2024-09-13T15:42:30.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.819+0000 7f81b14ef640 1 Processor -- start 2024-09-13T15:42:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.819+0000 7fed6975d640 1 Processor -- start 2024-09-13T15:42:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.819+0000 7f81b14ef640 1 -- start start 2024-09-13T15:42:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.819+0000 7fcee204f640 1 Processor -- start 2024-09-13T15:42:30.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.819+0000 7fed6975d640 1 -- start start 2024-09-13T15:42:30.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.819+0000 7fc4dbc78640 1 -- start start 2024-09-13T15:42:30.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.819+0000 7fcee204f640 1 -- start start 2024-09-13T15:42:30.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fed6975d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fed64095ab0 0x7fed64091590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fc4dbc78640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc4d40914f0 0x7fc4d40918c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fc4dbc78640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc4d4091e00 0x7fc4d412b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fc4dbc78640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d4092210 0x7fc4d412d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fed6975d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed64091ad0 0x7fed64091ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fed6975d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed640696d0 con 0x7fed6412b220 2024-09-13T15:42:30.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fed6975d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed64069f60 con 0x7fed64095ab0 2024-09-13T15:42:30.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fed6975d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed64092400 con 0x7fed64091ad0 2024-09-13T15:42:30.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7f81b14ef640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f81ac0986e0 0x7f81ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7f81b14ef640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f81ac0705a0 0x7f81ac12ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fed627fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed64091ad0 0x7fed64091ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fed627fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed64091ad0 0x7fed64091ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:44256/0 (socket says 172.21.15.70:44256) 2024-09-13T15:42:30.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fed627fc640 1 -- 172.21.15.70:0/735676879 learned_addr learned my addr 172.21.15.70:0/735676879 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fc4d9a14640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc4d40914f0 0x7fc4d40918c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fcee204f640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fcedc129e90 0x7fcedc12a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fcee204f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcedc0970e0 0x7fcedc097540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fcee204f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcedc099110 con 0x7fcedc12a7a0 2024-09-13T15:42:30.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fcee204f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc09a100 con 0x7fcedc0970e0 2024-09-13T15:42:30.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fcee204f640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc098b50 con 0x7fcedc129e90 2024-09-13T15:42:30.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fc4da215640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d4092210 0x7fc4d412d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fc4d9a14640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc4d40914f0 0x7fc4d40918c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41566/0 (socket says 172.21.15.70:41566) 2024-09-13T15:42:30.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fed62ffd640 1 --2- 172.21.15.70:0/735676879 >> v2:172.21.15.70:3300/0 conn(0x7fed64095ab0 0x7fed64091590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fc4da215640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d4092210 0x7fc4d412d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:44262/0 (socket says 172.21.15.70:44262) 2024-09-13T15:42:30.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fc4d9213640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc4d4091e00 0x7fc4d412b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fc4d9a14640 1 -- 172.21.15.70:0/770596471 learned_addr learned my addr 172.21.15.70:0/770596471 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.827 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fcedb7fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fcedc129e90 0x7fcedc12a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fc4dbc78640 1 -- 172.21.15.70:0/770596471 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d406e590 con 0x7fc4d4091e00 2024-09-13T15:42:30.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fcedb7fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fcedc129e90 0x7fcedc12a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41588/0 (socket says 172.21.15.70:41588) 2024-09-13T15:42:30.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fcedb7fe640 1 -- 172.21.15.70:0/2997016649 learned_addr learned my addr 172.21.15.70:0/2997016649 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fc4dbc78640 1 -- 172.21.15.70:0/770596471 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d4095820 con 0x7fc4d4092210 2024-09-13T15:42:30.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fc4dbc78640 1 -- 172.21.15.70:0/770596471 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d4093370 con 0x7fc4d40914f0 2024-09-13T15:42:30.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7f81b14ef640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac093e50 con 0x7f81ac0701d0 2024-09-13T15:42:30.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7f81aaffd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f81ac0986e0 0x7f81ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7f81aaffd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f81ac0986e0 0x7f81ac069d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41578/0 (socket says 172.21.15.70:41578) 2024-09-13T15:42:30.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7f81aaffd640 1 -- 172.21.15.70:0/4241762256 learned_addr learned my addr 172.21.15.70:0/4241762256 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7f81b14ef640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ac06bab0 con 0x7f81ac0705a0 2024-09-13T15:42:30.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fcedb7fe640 1 -- 172.21.15.70:0/2997016649 >> v2:172.21.15.70:3300/0 conn(0x7fcedc0970e0 msgr2=0x7fcedc097540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7f81b14ef640 1 -- 172.21.15.70:0/4241762256 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac06af00 con 0x7f81ac0986e0 2024-09-13T15:42:30.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fed627fc640 1 -- 172.21.15.70:0/735676879 >> v2:172.21.15.70:3300/0 conn(0x7fed64095ab0 msgr2=0x7fed64091590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fed627fc640 1 --2- 172.21.15.70:0/735676879 >> v2:172.21.15.70:3300/0 conn(0x7fed64095ab0 0x7fed64091590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fed627fc640 1 -- 172.21.15.70:0/735676879 >> v1:172.21.15.70:6789/0 conn(0x7fed6412b220 legacy=0x7fed6412d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fed627fc640 1 -- 172.21.15.70:0/735676879 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed64093590 con 0x7fed64091ad0 2024-09-13T15:42:30.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7f81a9ffb640 1 -- 172.21.15.70:0/4241762256 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273077503 0 0) 0x7f81ac093e50 con 0x7f81ac0701d0 2024-09-13T15:42:30.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7fc4da215640 1 -- 172.21.15.70:0/770596471 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc4d40914f0 msgr2=0x7fc4d40918c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7fc4da215640 1 --2- 172.21.15.70:0/770596471 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc4d40914f0 0x7fc4d40918c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7fc4da215640 1 -- 172.21.15.70:0/770596471 >> v2:172.21.15.70:3300/0 conn(0x7fc4d4091e00 msgr2=0x7fc4d412b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7fc4da215640 1 --2- 172.21.15.70:0/770596471 >> v2:172.21.15.70:3300/0 conn(0x7fc4d4091e00 0x7fc4d412b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7fc4da215640 1 -- 172.21.15.70:0/770596471 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d4094b30 con 0x7fc4d4092210 2024-09-13T15:42:30.832 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fcedb7fe640 1 --2- 172.21.15.70:0/2997016649 >> v2:172.21.15.70:3300/0 conn(0x7fcedc0970e0 0x7fcedc097540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7fc4d9a14640 1 --2- 172.21.15.70:0/770596471 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc4d40914f0 0x7fc4d40918c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:30.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7f81a9ffb640 1 -- 172.21.15.70:0/4241762256 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81940034e0 con 0x7f81ac0701d0 2024-09-13T15:42:30.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7f81ab7fe640 1 --2- 172.21.15.70:0/4241762256 >> v2:172.21.15.70:3300/0 conn(0x7f81ac0705a0 0x7f81ac12ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fcedb7fe640 1 -- 172.21.15.70:0/2997016649 >> v1:172.21.15.70:6789/0 conn(0x7fcedc12a7a0 legacy=0x7fcedc0969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.820+0000 7fcedb7fe640 1 -- 172.21.15.70:0/2997016649 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc0995c0 con 0x7fcedc129e90 2024-09-13T15:42:30.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7fc4d9213640 1 --2- 172.21.15.70:0/770596471 >> v2:172.21.15.70:3300/0 conn(0x7fc4d4091e00 0x7fc4d412b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:42:30.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7f81aaffd640 1 -- 172.21.15.70:0/4241762256 >> v2:172.21.15.70:3300/0 conn(0x7f81ac0705a0 msgr2=0x7f81ac12ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:30.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7f81aaffd640 1 --2- 172.21.15.70:0/4241762256 >> v2:172.21.15.70:3300/0 conn(0x7f81ac0705a0 0x7f81ac12ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7f81aaffd640 1 -- 172.21.15.70:0/4241762256 >> v1:172.21.15.70:6789/0 conn(0x7f81ac0701d0 legacy=0x7f81ac06a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7f81aaffd640 1 -- 172.21.15.70:0/4241762256 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac06b6e0 con 0x7f81ac0986e0 2024-09-13T15:42:30.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7f81ab7fe640 1 --2- 172.21.15.70:0/4241762256 >> v2:172.21.15.70:3300/0 conn(0x7f81ac0705a0 0x7f81ac12ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:30.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.821+0000 7fed627fc640 1 --2- 172.21.15.70:0/735676879 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed64091ad0 0x7fed64091ec0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fed5800bbc0 tx=0x7fed5800b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b6688bea39e048 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.822+0000 7fcedb7fe640 1 --2- 172.21.15.70:0/2997016649 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fcedc129e90 0x7fcedc12a260 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009910 tx=0x7fcecc0091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cc7a3633a52ed80 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.822+0000 7fed61ffb640 1 -- 172.21.15.70:0/735676879 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58032510 con 0x7fed64091ad0 2024-09-13T15:42:30.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.822+0000 7fc4da215640 1 --2- 172.21.15.70:0/770596471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d4092210 0x7fc4d412d5d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00b1b0 tx=0x7fc4cc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=678708610653c17a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.822+0000 7fed61ffb640 1 -- 172.21.15.70:0/735676879 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed58030050 con 0x7fed64091ad0 2024-09-13T15:42:30.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.822+0000 7fed61ffb640 1 -- 172.21.15.70:0/735676879 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58032900 con 0x7fed64091ad0 2024-09-13T15:42:30.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.822+0000 7fceda7fc640 1 -- 172.21.15.70:0/2997016649 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc02aca0 con 0x7fcedc129e90 2024-09-13T15:42:30.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.822+0000 7fceda7fc640 1 -- 172.21.15.70:0/2997016649 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc02ae00 con 0x7fcedc129e90 2024-09-13T15:42:30.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.822+0000 7fceda7fc640 1 -- 172.21.15.70:0/2997016649 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc032670 con 0x7fcedc129e90 2024-09-13T15:42:30.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.822+0000 7fc4d8a12640 1 -- 172.21.15.70:0/770596471 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0313f0 con 0x7fc4d4092210 2024-09-13T15:42:30.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.822+0000 7fc4d8a12640 1 -- 172.21.15.70:0/770596471 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4cc031550 con 0x7fc4d4092210 2024-09-13T15:42:30.838 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.822+0000 7fc4d8a12640 1 -- 172.21.15.70:0/770596471 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0318f0 con 0x7fc4d4092210 2024-09-13T15:42:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.822+0000 7f81aaffd640 1 --2- 172.21.15.70:0/4241762256 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f81ac0986e0 0x7f81ac069d80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f819c009910 tx=0x7f819c0091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4502893fbef8783d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.822+0000 7f81a9ffb640 1 -- 172.21.15.70:0/4241762256 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c02ad20 con 0x7f81ac0986e0 2024-09-13T15:42:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.822+0000 7f81a9ffb640 1 -- 172.21.15.70:0/4241762256 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c02ae80 con 0x7f81ac0986e0 2024-09-13T15:42:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.822+0000 7f81a9ffb640 1 -- 172.21.15.70:0/4241762256 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c032670 con 0x7f81ac0986e0 2024-09-13T15:42:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fcee204f640 1 -- 172.21.15.70:0/2997016649 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fcedc129e90 msgr2=0x7fcedc12a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fcee204f640 1 --2- 172.21.15.70:0/2997016649 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fcedc129e90 0x7fcedc12a260 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009910 tx=0x7fcecc0091d0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fed6975d640 1 -- 172.21.15.70:0/735676879 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed64091ad0 msgr2=0x7fed64091ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fed6975d640 1 --2- 172.21.15.70:0/735676879 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed64091ad0 0x7fed64091ec0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fed5800bbc0 tx=0x7fed5800b460 comp rx=0 tx=0).stop 2024-09-13T15:42:30.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fcee204f640 1 -- 172.21.15.70:0/2997016649 shutdown_connections 2024-09-13T15:42:30.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fcee204f640 1 --2- 172.21.15.70:0/2997016649 >> v2:172.21.15.70:3300/0 conn(0x7fcedc0970e0 0x7fcedc097540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.840 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fcee204f640 1 --2- 172.21.15.70:0/2997016649 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fcedc129e90 0x7fcedc12a260 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fcee204f640 1 -- 172.21.15.70:0/2997016649 >> 172.21.15.70:0/2997016649 conn(0x7fcedc09d840 msgr2=0x7fcedc09dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fed6975d640 1 -- 172.21.15.70:0/735676879 shutdown_connections 2024-09-13T15:42:30.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fcee204f640 1 -- 172.21.15.70:0/2997016649 shutdown_connections 2024-09-13T15:42:30.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fed6975d640 1 --2- 172.21.15.70:0/735676879 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed64091ad0 0x7fed64091ec0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.841 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fed6975d640 1 --2- 172.21.15.70:0/735676879 >> v2:172.21.15.70:3300/0 conn(0x7fed64095ab0 0x7fed64091590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fc4dbc78640 1 -- 172.21.15.70:0/770596471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d4092210 msgr2=0x7fc4d412d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fed6975d640 1 -- 172.21.15.70:0/735676879 >> 172.21.15.70:0/735676879 conn(0x7fed6409aca0 msgr2=0x7fed64096ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fc4dbc78640 1 --2- 172.21.15.70:0/770596471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d4092210 0x7fc4d412d5d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc00b1b0 tx=0x7fc4cc009780 comp rx=0 tx=0).stop 2024-09-13T15:42:30.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fcee204f640 1 -- 172.21.15.70:0/2997016649 wait complete. 2024-09-13T15:42:30.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fed6975d640 1 -- 172.21.15.70:0/735676879 shutdown_connections 2024-09-13T15:42:30.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fc4dbc78640 1 -- 172.21.15.70:0/770596471 shutdown_connections 2024-09-13T15:42:30.843 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fed6975d640 1 -- 172.21.15.70:0/735676879 wait complete. 2024-09-13T15:42:30.844 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fc4dbc78640 1 --2- 172.21.15.70:0/770596471 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d4092210 0x7fc4d412d5d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.844 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fc4dbc78640 1 --2- 172.21.15.70:0/770596471 >> v2:172.21.15.70:3300/0 conn(0x7fc4d4091e00 0x7fc4d412b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.844 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fc4dbc78640 1 --2- 172.21.15.70:0/770596471 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc4d40914f0 0x7fc4d40918c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.844 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fc4dbc78640 1 -- 172.21.15.70:0/770596471 >> 172.21.15.70:0/770596471 conn(0x7fc4d40959f0 msgr2=0x7fc4d4095df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.844 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fc4dbc78640 1 -- 172.21.15.70:0/770596471 shutdown_connections 2024-09-13T15:42:30.844 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7fc4dbc78640 1 -- 172.21.15.70:0/770596471 wait complete. 2024-09-13T15:42:30.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7f81b14ef640 1 -- 172.21.15.70:0/4241762256 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f81ac0986e0 msgr2=0x7f81ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.845 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7f81b14ef640 1 --2- 172.21.15.70:0/4241762256 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f81ac0986e0 0x7f81ac069d80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f819c009910 tx=0x7f819c0091d0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7f81b14ef640 1 -- 172.21.15.70:0/4241762256 shutdown_connections 2024-09-13T15:42:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7f81b14ef640 1 --2- 172.21.15.70:0/4241762256 >> v2:172.21.15.70:3300/0 conn(0x7f81ac0705a0 0x7f81ac12ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7f81b14ef640 1 --2- 172.21.15.70:0/4241762256 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f81ac0986e0 0x7f81ac069d80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7f81b14ef640 1 -- 172.21.15.70:0/4241762256 >> 172.21.15.70:0/4241762256 conn(0x7f81ac09a3c0 msgr2=0x7f81ac0962c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7f81b14ef640 1 -- 172.21.15.70:0/4241762256 shutdown_connections 2024-09-13T15:42:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.823+0000 7f81b14ef640 1 -- 172.21.15.70:0/4241762256 wait complete. 2024-09-13T15:42:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.824+0000 7fcee204f640 1 Processor -- start 2024-09-13T15:42:30.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.824+0000 7fc4dbc78640 1 Processor -- start 2024-09-13T15:42:30.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.824+0000 7fcee204f640 1 -- start start 2024-09-13T15:42:30.847 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.824+0000 7fc4dbc78640 1 -- start start 2024-09-13T15:42:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.824+0000 7fed6975d640 1 Processor -- start 2024-09-13T15:42:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.824+0000 7fed6975d640 1 -- start start 2024-09-13T15:42:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.824+0000 7fc4dbc78640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc4d40914f0 0x7fc4d4136070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.824+0000 7f81b14ef640 1 Processor -- start 2024-09-13T15:42:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7f81b14ef640 1 -- start start 2024-09-13T15:42:30.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4dbc78640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4d4091e00 0x7fc4d41365b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fcee204f640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fcedc0970e0 0x7fcedc119b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fed6975d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed64091ad0 0x7fed64125c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.849 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fed6975d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fed64095ab0 0x7fed6411f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.850 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4dbc78640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d4092210 0x7fc4d408d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.850 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4dbc78640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d411e250 con 0x7fc4d4091e00 2024-09-13T15:42:30.850 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fcee204f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcedc129e90 0x7fcedc11a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.850 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4dbc78640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d412dc30 con 0x7fc4d4092210 2024-09-13T15:42:30.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fcee204f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc12a7a0 0x7fcedc11a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fcee204f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc11c930 con 0x7fcedc12a7a0 2024-09-13T15:42:30.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4dbc78640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d411ea30 con 0x7fc4d40914f0 2024-09-13T15:42:30.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4d9213640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4d4091e00 0x7fc4d41365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4d9a14640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc4d40914f0 0x7fc4d4136070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.851 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4d9213640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4d4091e00 0x7fc4d41365b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43852/0 (socket says 172.21.15.70:43852) 2024-09-13T15:42:30.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4d9213640 1 -- 172.21.15.70:0/517758175 learned_addr learned my addr 172.21.15.70:0/517758175 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4da215640 1 --2- 172.21.15.70:0/517758175 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d4092210 0x7fc4d408d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fcedb7fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fcedc0970e0 0x7fcedc119b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fed6975d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed6412b220 0x7fed6411f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.852 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fed627fc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fed64095ab0 0x7fed6411f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fed627fc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fed64095ab0 0x7fed6411f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41608/0 (socket says 172.21.15.70:41608) 2024-09-13T15:42:30.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fed6975d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed64120b50 con 0x7fed64091ad0 2024-09-13T15:42:30.853 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fed627fc640 1 -- 172.21.15.70:0/3028391896 learned_addr learned my addr 172.21.15.70:0/3028391896 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4d9213640 1 -- 172.21.15.70:0/517758175 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc4d40914f0 msgr2=0x7fc4d4136070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4d9213640 1 --2- 172.21.15.70:0/517758175 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc4d40914f0 0x7fc4d4136070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fed6975d640 1 -- 172.21.15.70:0/3028391896 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed64121c00 con 0x7fed6412b220 2024-09-13T15:42:30.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fed637fe640 1 --2- 172.21.15.70:0/3028391896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed6412b220 0x7fed6411f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fed62ffd640 1 --2- 172.21.15.70:0/3028391896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed64091ad0 0x7fed64125c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4d9213640 1 -- 172.21.15.70:0/517758175 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d4092210 msgr2=0x7fc4d408d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.855 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fed6975d640 1 -- 172.21.15.70:0/3028391896 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed641251c0 con 0x7fed64095ab0 2024-09-13T15:42:30.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4d9213640 1 --2- 172.21.15.70:0/517758175 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d4092210 0x7fc4d408d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4d9213640 1 -- 172.21.15.70:0/517758175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d0009450 con 0x7fc4d4091e00 2024-09-13T15:42:30.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fcedb7fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fcedc0970e0 0x7fcedc119b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41606/0 (socket says 172.21.15.70:41606) 2024-09-13T15:42:30.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fcedb7fe640 1 -- 172.21.15.70:0/2915088746 learned_addr learned my addr 172.21.15.70:0/2915088746 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fcedbfff640 1 --2- 172.21.15.70:0/2915088746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc12a7a0 0x7fcedc11a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fcedaffd640 1 --2- 172.21.15.70:0/2915088746 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcedc129e90 0x7fcedc11a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fc4d9213640 1 --2- 172.21.15.70:0/517758175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4d4091e00 0x7fc4d41365b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400d860 tx=0x7fc4c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.825+0000 7fcee204f640 1 -- 172.21.15.70:0/2915088746 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc11b680 con 0x7fcedc129e90 2024-09-13T15:42:30.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fcee204f640 1 -- 172.21.15.70:0/2915088746 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc11c6f0 con 0x7fcedc0970e0 2024-09-13T15:42:30.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fc4c9ffb640 1 -- 172.21.15.70:0/517758175 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400de50 con 0x7fc4d4091e00 2024-09-13T15:42:30.858 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fc4c9ffb640 1 -- 172.21.15.70:0/517758175 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c400b870 con 0x7fc4d4091e00 2024-09-13T15:42:30.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7f81b14ef640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f81ac0701d0 0x7f81ac139320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7f81b14ef640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81ac0705a0 0x7f81ac139860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7f81b14ef640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ac139da0 0x7f81ac08c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:30.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7f81b14ef640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac08dfb0 con 0x7f81ac139da0 2024-09-13T15:42:30.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7f81b14ef640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac08fc20 con 0x7f81ac0705a0 2024-09-13T15:42:30.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7f81b14ef640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac08f850 con 0x7f81ac0701d0 2024-09-13T15:42:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fc4dbc78640 1 -- 172.21.15.70:0/517758175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4cc009a20 con 0x7fc4d4091e00 2024-09-13T15:42:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fc4dbc78640 1 -- 172.21.15.70:0/517758175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d408f180 con 0x7fc4d4091e00 2024-09-13T15:42:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fed627fc640 1 -- 172.21.15.70:0/3028391896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed6412b220 msgr2=0x7fed6411f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fed627fc640 1 --2- 172.21.15.70:0/3028391896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed6412b220 0x7fed6411f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fed627fc640 1 -- 172.21.15.70:0/3028391896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed64091ad0 msgr2=0x7fed64125c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fed627fc640 1 --2- 172.21.15.70:0/3028391896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed64091ad0 0x7fed64125c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fed627fc640 1 -- 172.21.15.70:0/3028391896 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5800b700 con 0x7fed64095ab0 2024-09-13T15:42:30.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fed62ffd640 1 --2- 172.21.15.70:0/3028391896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed64091ad0 0x7fed64125c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:42:30.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fed627fc640 1 --2- 172.21.15.70:0/3028391896 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fed64095ab0 0x7fed6411f180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fed5800b4a0 tx=0x7fed5803a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fcedaffd640 1 -- 172.21.15.70:0/2915088746 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fcedc0970e0 msgr2=0x7fcedc119b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fcedaffd640 1 --2- 172.21.15.70:0/2915088746 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fcedc0970e0 0x7fcedc119b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7f81aa7fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81ac0705a0 0x7f81ac139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fcedaffd640 1 -- 172.21.15.70:0/2915088746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc12a7a0 msgr2=0x7fcedc11a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fcedaffd640 1 --2- 172.21.15.70:0/2915088746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc12a7a0 0x7fcedc11a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7f81aa7fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81ac0705a0 0x7f81ac139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:44296/0 (socket says 172.21.15.70:44296) 2024-09-13T15:42:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7f81aa7fc640 1 -- 172.21.15.70:0/208713347 learned_addr learned my addr 172.21.15.70:0/208713347 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:30.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fed4affd640 1 -- 172.21.15.70:0/3028391896 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58034400 con 0x7fed64095ab0 2024-09-13T15:42:30.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fed4affd640 1 -- 172.21.15.70:0/3028391896 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed580345c0 con 0x7fed64095ab0 2024-09-13T15:42:30.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fed4affd640 1 -- 172.21.15.70:0/3028391896 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58034980 con 0x7fed64095ab0 2024-09-13T15:42:30.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.826+0000 7fcedaffd640 1 -- 172.21.15.70:0/2915088746 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc009450 con 0x7fcedc129e90 2024-09-13T15:42:30.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7f81aaffd640 1 --2- 172.21.15.70:0/208713347 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f81ac0701d0 0x7f81ac139320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7f81ab7fe640 1 --2- 172.21.15.70:0/208713347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ac139da0 0x7f81ac08c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:30.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7fcedaffd640 1 --2- 172.21.15.70:0/2915088746 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcedc129e90 0x7fcedc11a090 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009fd0 tx=0x7fcec4009060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7fced37fe640 1 -- 172.21.15.70:0/2915088746 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec400f510 con 0x7fcedc129e90 2024-09-13T15:42:30.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7fed6975d640 1 -- 172.21.15.70:0/3028391896 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed64120640 con 0x7fed64095ab0 2024-09-13T15:42:30.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7f81aa7fc640 1 -- 172.21.15.70:0/208713347 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f81ac0701d0 msgr2=0x7f81ac139320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7f81aa7fc640 1 --2- 172.21.15.70:0/208713347 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f81ac0701d0 0x7f81ac139320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7f81aa7fc640 1 -- 172.21.15.70:0/208713347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ac139da0 msgr2=0x7f81ac08c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7f81aa7fc640 1 --2- 172.21.15.70:0/208713347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ac139da0 0x7f81ac08c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7f81aa7fc640 1 -- 172.21.15.70:0/208713347 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a4009a20 con 0x7f81ac0705a0 2024-09-13T15:42:30.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7f81ab7fe640 1 --2- 172.21.15.70:0/208713347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ac139da0 0x7f81ac08c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:30.865 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7f81aaffd640 1 --2- 172.21.15.70:0/208713347 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f81ac0701d0 0x7f81ac139320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7fc4dbc78640 1 -- 172.21.15.70:0/517758175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d405ed70 con 0x7fc4d4091e00 2024-09-13T15:42:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.828+0000 7fc4c9ffb640 1 -- 172.21.15.70:0/517758175 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400bbf0 con 0x7fc4d4091e00 2024-09-13T15:42:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.828+0000 7fed6975d640 1 -- 172.21.15.70:0/3028391896 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed64121280 con 0x7fed64095ab0 2024-09-13T15:42:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.828+0000 7f81aa7fc640 1 --2- 172.21.15.70:0/208713347 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81ac0705a0 0x7f81ac139860 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f81a000c4e0 tx=0x7f81a0007d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:30.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7fced37fe640 1 -- 172.21.15.70:0/2915088746 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec400f670 con 0x7fcedc129e90 2024-09-13T15:42:30.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7fcee204f640 1 -- 172.21.15.70:0/2915088746 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc11ac00 con 0x7fcedc129e90 2024-09-13T15:42:30.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.827+0000 7fcee204f640 1 -- 172.21.15.70:0/2915088746 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc139410 con 0x7fcedc129e90 2024-09-13T15:42:30.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.828+0000 7fced37fe640 1 -- 172.21.15.70:0/2915088746 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec400f510 con 0x7fcedc129e90 2024-09-13T15:42:30.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.828+0000 7f8192ffd640 1 -- 172.21.15.70:0/208713347 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0016070 con 0x7f81ac0705a0 2024-09-13T15:42:30.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.828+0000 7f8192ffd640 1 -- 172.21.15.70:0/208713347 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a0011590 con 0x7f81ac0705a0 2024-09-13T15:42:30.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.828+0000 7f8192ffd640 1 -- 172.21.15.70:0/208713347 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0011990 con 0x7f81ac0705a0 2024-09-13T15:42:30.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.828+0000 7f81b14ef640 1 -- 172.21.15.70:0/208713347 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819c009450 con 0x7f81ac0705a0 2024-09-13T15:42:30.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.828+0000 7f81b14ef640 1 -- 172.21.15.70:0/208713347 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ac08c930 con 0x7f81ac0705a0 2024-09-13T15:42:30.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.829+0000 7fc4c9ffb640 1 -- 172.21.15.70:0/517758175 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fc4c401e460 con 0x7fc4d4091e00 2024-09-13T15:42:30.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.829+0000 7fed6975d640 1 -- 172.21.15.70:0/3028391896 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6406e8e0 con 0x7fed64095ab0 2024-09-13T15:42:30.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.829+0000 7fcee204f640 1 -- 172.21.15.70:0/2915088746 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc05ed70 con 0x7fcedc129e90 2024-09-13T15:42:30.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.829+0000 7fed4affd640 1 -- 172.21.15.70:0/3028391896 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fed58034d30 con 0x7fed64095ab0 2024-09-13T15:42:30.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.830+0000 7fc4c9ffb640 1 -- 172.21.15.70:0/517758175 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fc4c4042fe0 con 0x7fc4d4091e00 2024-09-13T15:42:30.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.830+0000 7fed4affd640 1 -- 172.21.15.70:0/3028391896 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fed58088470 con 0x7fed64095ab0 2024-09-13T15:42:30.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.830+0000 7fced37fe640 1 -- 172.21.15.70:0/2915088746 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fcec400faf0 con 0x7fcedc129e90 2024-09-13T15:42:30.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.830+0000 7f81b14ef640 1 -- 172.21.15.70:0/208713347 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ac05ed70 con 0x7f81ac0705a0 2024-09-13T15:42:30.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.830+0000 7fced37fe640 1 -- 172.21.15.70:0/2915088746 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fcec4041460 con 0x7fcedc129e90 2024-09-13T15:42:30.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.830+0000 7f8192ffd640 1 -- 172.21.15.70:0/208713347 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f81a0012420 con 0x7f81ac0705a0 2024-09-13T15:42:30.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.831+0000 7f8192ffd640 1 -- 172.21.15.70:0/208713347 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f81a0062bd0 con 0x7f81ac0705a0 2024-09-13T15:42:30.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.833+0000 7fc4c9ffb640 1 -- 172.21.15.70:0/517758175 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fc4c403f430 con 0x7fc4d4091e00 2024-09-13T15:42:30.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.834+0000 7fed4affd640 1 -- 172.21.15.70:0/3028391896 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fed58066f20 con 0x7fed64095ab0 2024-09-13T15:42:30.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.834+0000 7f8192ffd640 1 -- 172.21.15.70:0/208713347 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f81a003dcb0 con 0x7f81ac0705a0 2024-09-13T15:42:30.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.840+0000 7fced37fe640 1 -- 172.21.15.70:0/2915088746 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fcec403e1e0 con 0x7fcedc129e90 2024-09-13T15:42:30.972 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.970+0000 7fc4dbc78640 1 -- 172.21.15.70:0/517758175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc4d408fdf0 con 0x7fc4d4091e00 2024-09-13T15:42:30.972 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.970+0000 7fc4c9ffb640 1 -- 172.21.15.70:0/517758175 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc4c4042c40 con 0x7fc4d4091e00 2024-09-13T15:42:30.972 INFO:teuthology.orchestra.run.smithi070.stdout:38654705667 2024-09-13T15:42:30.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.972+0000 7fc4dbc78640 1 -- 172.21.15.70:0/517758175 >> v1:172.21.15.70:6816/32242 conn(0x7fc4b804cb70 legacy=0x7fc4b804f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.972+0000 7fc4dbc78640 1 -- 172.21.15.70:0/517758175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4d4091e00 msgr2=0x7fc4d41365b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.972+0000 7fc4dbc78640 1 --2- 172.21.15.70:0/517758175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4d4091e00 0x7fc4d41365b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400d860 tx=0x7fc4c4008040 comp rx=0 tx=0).stop 2024-09-13T15:42:30.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.973+0000 7fc4dbc78640 1 -- 172.21.15.70:0/517758175 shutdown_connections 2024-09-13T15:42:30.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.973+0000 7fc4dbc78640 1 --2- 172.21.15.70:0/517758175 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc4d4092210 0x7fc4d408d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.973+0000 7fc4dbc78640 1 --2- 172.21.15.70:0/517758175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4d4091e00 0x7fc4d41365b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.973+0000 7fc4dbc78640 1 --2- 172.21.15.70:0/517758175 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc4d40914f0 0x7fc4d4136070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.973+0000 7fc4dbc78640 1 -- 172.21.15.70:0/517758175 >> 172.21.15.70:0/517758175 conn(0x7fc4d40959f0 msgr2=0x7fc4d406cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.973+0000 7fc4dbc78640 1 -- 172.21.15.70:0/517758175 shutdown_connections 2024-09-13T15:42:30.974 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.973+0000 7fc4dbc78640 1 -- 172.21.15.70:0/517758175 wait complete. 2024-09-13T15:42:30.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.975+0000 7fed6975d640 1 -- 172.21.15.70:0/3028391896 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fed6405ed70 con 0x7fed64095ab0 2024-09-13T15:42:30.976 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.975+0000 7fed4affd640 1 -- 172.21.15.70:0/3028391896 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fed58066d40 con 0x7fed64095ab0 2024-09-13T15:42:30.976 INFO:teuthology.orchestra.run.smithi070.stdout:38654705667 2024-09-13T15:42:30.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.976+0000 7f81b14ef640 1 -- 172.21.15.70:0/208713347 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f81ac000900 con 0x7f81ac0705a0 2024-09-13T15:42:30.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.976+0000 7f8192ffd640 1 -- 172.21.15.70:0/208713347 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f81a00414c0 con 0x7f81ac0705a0 2024-09-13T15:42:30.977 INFO:teuthology.orchestra.run.smithi070.stdout:38654705667 2024-09-13T15:42:30.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.977+0000 7fed6975d640 1 -- 172.21.15.70:0/3028391896 >> v1:172.21.15.70:6816/32242 conn(0x7fed5004cbf0 legacy=0x7fed5004f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.977+0000 7fed6975d640 1 -- 172.21.15.70:0/3028391896 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fed64095ab0 msgr2=0x7fed6411f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.977+0000 7fed6975d640 1 --2- 172.21.15.70:0/3028391896 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fed64095ab0 0x7fed6411f180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fed5800b4a0 tx=0x7fed5803a000 comp rx=0 tx=0).stop 2024-09-13T15:42:30.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.977+0000 7fed6975d640 1 -- 172.21.15.70:0/3028391896 shutdown_connections 2024-09-13T15:42:30.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.977+0000 7fed6975d640 1 --2- 172.21.15.70:0/3028391896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fed6412b220 0x7fed6411f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.977+0000 7fed6975d640 1 --2- 172.21.15.70:0/3028391896 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fed64095ab0 0x7fed6411f180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.977+0000 7fed6975d640 1 --2- 172.21.15.70:0/3028391896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed64091ad0 0x7fed64125c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.977+0000 7fed6975d640 1 -- 172.21.15.70:0/3028391896 >> 172.21.15.70:0/3028391896 conn(0x7fed6409aca0 msgr2=0x7fed64097fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.977+0000 7fed6975d640 1 -- 172.21.15.70:0/3028391896 shutdown_connections 2024-09-13T15:42:30.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.977+0000 7fed6975d640 1 -- 172.21.15.70:0/3028391896 wait complete. 2024-09-13T15:42:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.979+0000 7f81b14ef640 1 -- 172.21.15.70:0/208713347 >> v1:172.21.15.70:6816/32242 conn(0x7f819804ccc0 legacy=0x7f819804f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.979+0000 7f81b14ef640 1 -- 172.21.15.70:0/208713347 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81ac0705a0 msgr2=0x7f81ac139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.979+0000 7f81b14ef640 1 --2- 172.21.15.70:0/208713347 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81ac0705a0 0x7f81ac139860 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f81a000c4e0 tx=0x7f81a0007d80 comp rx=0 tx=0).stop 2024-09-13T15:42:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.979+0000 7f81b14ef640 1 -- 172.21.15.70:0/208713347 shutdown_connections 2024-09-13T15:42:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.979+0000 7f81b14ef640 1 --2- 172.21.15.70:0/208713347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81ac139da0 0x7f81ac08c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.979+0000 7f81b14ef640 1 --2- 172.21.15.70:0/208713347 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81ac0705a0 0x7f81ac139860 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.979+0000 7f81b14ef640 1 --2- 172.21.15.70:0/208713347 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f81ac0701d0 0x7f81ac139320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.979+0000 7f81b14ef640 1 -- 172.21.15.70:0/208713347 >> 172.21.15.70:0/208713347 conn(0x7f81ac09a3c0 msgr2=0x7f81ac06dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.979+0000 7f81b14ef640 1 -- 172.21.15.70:0/208713347 shutdown_connections 2024-09-13T15:42:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.979+0000 7f81b14ef640 1 -- 172.21.15.70:0/208713347 wait complete. 2024-09-13T15:42:30.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.984+0000 7fcee204f640 1 -- 172.21.15.70:0/2915088746 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcedc128a60 con 0x7fcedc129e90 2024-09-13T15:42:30.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.984+0000 7fced37fe640 1 -- 172.21.15.70:0/2915088746 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcec4045020 con 0x7fcedc129e90 2024-09-13T15:42:30.985 INFO:teuthology.orchestra.run.smithi070.stdout:38654705666 2024-09-13T15:42:30.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.2 2024-09-13T15:42:30.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.987+0000 7fcee204f640 1 -- 172.21.15.70:0/2915088746 >> v1:172.21.15.70:6816/32242 conn(0x7fcebc04cbd0 legacy=0x7fcebc04f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.987+0000 7fcee204f640 1 -- 172.21.15.70:0/2915088746 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcedc129e90 msgr2=0x7fcedc11a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:30.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.987+0000 7fcee204f640 1 --2- 172.21.15.70:0/2915088746 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcedc129e90 0x7fcedc11a090 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009fd0 tx=0x7fcec4009060 comp rx=0 tx=0).stop 2024-09-13T15:42:30.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.987+0000 7fcee204f640 1 -- 172.21.15.70:0/2915088746 shutdown_connections 2024-09-13T15:42:30.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.987+0000 7fcee204f640 1 --2- 172.21.15.70:0/2915088746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc12a7a0 0x7fcedc11a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.987+0000 7fcee204f640 1 --2- 172.21.15.70:0/2915088746 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcedc129e90 0x7fcedc11a090 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.987+0000 7fcee204f640 1 --2- 172.21.15.70:0/2915088746 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fcedc0970e0 0x7fcedc119b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:30.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.987+0000 7fcee204f640 1 -- 172.21.15.70:0/2915088746 >> 172.21.15.70:0/2915088746 conn(0x7fcedc09d840 msgr2=0x7fcedc09b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:30.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.987+0000 7fcee204f640 1 -- 172.21.15.70:0/2915088746 shutdown_connections 2024-09-13T15:42:30.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:30.987+0000 7fcee204f640 1 -- 172.21.15.70:0/2915088746 wait complete. 2024-09-13T15:42:30.992 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.0 2024-09-13T15:42:30.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.3 2024-09-13T15:42:31.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.1 2024-09-13T15:42:31.987 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T15:42:31.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T15:42:31.995 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T15:42:32.000 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T15:42:32.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.095+0000 7f627b15d640 1 Processor -- start 2024-09-13T15:42:32.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.095+0000 7f627b15d640 1 -- start start 2024-09-13T15:42:32.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.095+0000 7f627b15d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6274096bc0 0x7f627408f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.095+0000 7f627b15d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f627408f820 0x7f627408fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.095+0000 7f627b15d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f627412c330 0x7f627412e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.095+0000 7f627b15d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6274090420 con 0x7f627412c330 2024-09-13T15:42:32.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.095+0000 7f627b15d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6274090150 con 0x7f6274096bc0 2024-09-13T15:42:32.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.095+0000 7f627b15d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6274091410 con 0x7f627408f820 2024-09-13T15:42:32.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.095+0000 7f62796fa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f627412c330 0x7f627412e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.096+0000 7f62796fa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f627412c330 0x7f627412e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43898/0 (socket says 172.21.15.70:43898) 2024-09-13T15:42:32.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.096+0000 7f62796fa640 1 -- 172.21.15.70:0/2684319160 learned_addr learned my addr 172.21.15.70:0/2684319160 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:32.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.096+0000 7f6273fff640 1 --2- 172.21.15.70:0/2684319160 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f627408f820 0x7f627408fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.096+0000 7f6278ef9640 1 --2- 172.21.15.70:0/2684319160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6274096bc0 0x7f627408f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.096+0000 7f6273fff640 1 -- 172.21.15.70:0/2684319160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6274096bc0 msgr2=0x7f627408f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:32.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.096+0000 7f6273fff640 1 --2- 172.21.15.70:0/2684319160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6274096bc0 0x7f627408f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.096+0000 7f6273fff640 1 -- 172.21.15.70:0/2684319160 >> v2:172.21.15.70:3300/0 conn(0x7f627412c330 msgr2=0x7f627412e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:32.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.096+0000 7f6273fff640 1 --2- 172.21.15.70:0/2684319160 >> v2:172.21.15.70:3300/0 conn(0x7f627412c330 0x7f627412e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.096+0000 7f6273fff640 1 -- 172.21.15.70:0/2684319160 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6274090fd0 con 0x7f627408f820 2024-09-13T15:42:32.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.096+0000 7f62796fa640 1 --2- 172.21.15.70:0/2684319160 >> v2:172.21.15.70:3300/0 conn(0x7f627412c330 0x7f627412e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.096+0000 7f6278ef9640 1 --2- 172.21.15.70:0/2684319160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6274096bc0 0x7f627408f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:42:32.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.097+0000 7f6273fff640 1 --2- 172.21.15.70:0/2684319160 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f627408f820 0x7f627408fc10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6260009850 tx=0x7f62600090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afd837b0f9026dba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:32.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.097+0000 7f62737fe640 1 -- 172.21.15.70:0/2684319160 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626002abc0 con 0x7f627408f820 2024-09-13T15:42:32.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.097+0000 7f62737fe640 1 -- 172.21.15.70:0/2684319160 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626002ad20 con 0x7f627408f820 2024-09-13T15:42:32.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.097+0000 7f62737fe640 1 -- 172.21.15.70:0/2684319160 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260032670 con 0x7f627408f820 2024-09-13T15:42:32.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.098+0000 7f627b15d640 1 -- 172.21.15.70:0/2684319160 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f627408f820 msgr2=0x7f627408fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.098+0000 7f627b15d640 1 --2- 172.21.15.70:0/2684319160 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f627408f820 0x7f627408fc10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6260009850 tx=0x7f62600090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.098+0000 7f627b15d640 1 -- 172.21.15.70:0/2684319160 shutdown_connections 2024-09-13T15:42:32.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.098+0000 7f627b15d640 1 --2- 172.21.15.70:0/2684319160 >> v2:172.21.15.70:3300/0 conn(0x7f627412c330 0x7f627412e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.098+0000 7f627b15d640 1 --2- 172.21.15.70:0/2684319160 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f627408f820 0x7f627408fc10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.098+0000 7f627b15d640 1 --2- 172.21.15.70:0/2684319160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6274096bc0 0x7f627408f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.098+0000 7f627b15d640 1 -- 172.21.15.70:0/2684319160 >> 172.21.15.70:0/2684319160 conn(0x7f6274097cb0 msgr2=0x7f62740980b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:32.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.098+0000 7f627b15d640 1 -- 172.21.15.70:0/2684319160 shutdown_connections 2024-09-13T15:42:32.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.098+0000 7f627b15d640 1 -- 172.21.15.70:0/2684319160 wait complete. 2024-09-13T15:42:32.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.100+0000 7f627b15d640 1 Processor -- start 2024-09-13T15:42:32.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.100+0000 7f627b15d640 1 -- start start 2024-09-13T15:42:32.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.100+0000 7f627b15d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f627408f820 0x7f6274087f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.100+0000 7f627b15d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6274096bc0 0x7f62740884a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.100+0000 7f627b15d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f627412c330 0x7f62740889e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.100+0000 7f627b15d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6274094ab0 con 0x7f627412c330 2024-09-13T15:42:32.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.100+0000 7f627b15d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627408ab70 con 0x7f627408f820 2024-09-13T15:42:32.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.100+0000 7f627b15d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6274089c40 con 0x7f6274096bc0 2024-09-13T15:42:32.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.101+0000 7f6273fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6274096bc0 0x7f62740884a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.101+0000 7f6273fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6274096bc0 0x7f62740884a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41638/0 (socket says 172.21.15.70:41638) 2024-09-13T15:42:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.101+0000 7f6273fff640 1 -- 172.21.15.70:0/927758294 learned_addr learned my addr 172.21.15.70:0/927758294 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.101+0000 7f6278ef9640 1 --2- 172.21.15.70:0/927758294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f627408f820 0x7f6274087f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.101+0000 7f62796fa640 1 --2- 172.21.15.70:0/927758294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f627412c330 0x7f62740889e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.101+0000 7f6273fff640 1 -- 172.21.15.70:0/927758294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f627408f820 msgr2=0x7f6274087f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.101+0000 7f6273fff640 1 --2- 172.21.15.70:0/927758294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f627408f820 0x7f6274087f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.101+0000 7f6273fff640 1 -- 172.21.15.70:0/927758294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f627412c330 msgr2=0x7f62740889e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.101+0000 7f6273fff640 1 --2- 172.21.15.70:0/927758294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f627412c330 0x7f62740889e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.101+0000 7f6273fff640 1 -- 172.21.15.70:0/927758294 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626c009ae0 con 0x7f6274096bc0 2024-09-13T15:42:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.101+0000 7f62796fa640 1 --2- 172.21.15.70:0/927758294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f627412c330 0x7f62740889e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.101+0000 7f6278ef9640 1 --2- 172.21.15.70:0/927758294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f627408f820 0x7f6274087f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.101+0000 7f6273fff640 1 --2- 172.21.15.70:0/927758294 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6274096bc0 0x7f62740884a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f626002a4f0 tx=0x7f6260032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.102+0000 7f6270ff9640 1 -- 172.21.15.70:0/927758294 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626003d720 con 0x7f6274096bc0 2024-09-13T15:42:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.102+0000 7f6270ff9640 1 -- 172.21.15.70:0/927758294 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626002ae90 con 0x7f6274096bc0 2024-09-13T15:42:32.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.102+0000 7f6270ff9640 1 -- 172.21.15.70:0/927758294 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626003da90 con 0x7f6274096bc0 2024-09-13T15:42:32.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.102+0000 7f627b15d640 1 -- 172.21.15.70:0/927758294 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6260009390 con 0x7f6274096bc0 2024-09-13T15:42:32.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.102+0000 7f627b15d640 1 -- 172.21.15.70:0/927758294 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627406f100 con 0x7f6274096bc0 2024-09-13T15:42:32.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.103+0000 7f627b15d640 1 -- 172.21.15.70:0/927758294 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627406f3b0 con 0x7f6274096bc0 2024-09-13T15:42:32.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.103+0000 7f6270ff9640 1 -- 172.21.15.70:0/927758294 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f6260039040 con 0x7f6274096bc0 2024-09-13T15:42:32.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.104+0000 7f6270ff9640 1 -- 172.21.15.70:0/927758294 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f62600828d0 con 0x7f6274096bc0 2024-09-13T15:42:32.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.110+0000 7f6270ff9640 1 -- 172.21.15.70:0/927758294 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f626005da10 con 0x7f6274096bc0 2024-09-13T15:42:32.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.133+0000 7f09ae0ef640 1 Processor -- start 2024-09-13T15:42:32.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.133+0000 7f86feb9d640 1 Processor -- start 2024-09-13T15:42:32.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.133+0000 7f09ae0ef640 1 -- start start 2024-09-13T15:42:32.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.133+0000 7f86feb9d640 1 -- start start 2024-09-13T15:42:32.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f09ae0ef640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09a8129c60 0x7f09a812a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f09ae0ef640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f09a812a570 0x7f09a8128be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f86feb9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f86f80701d0 0x7f86f80987e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f86feb9d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86f8099160 0x7f86f8093720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f09ae0ef640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a80991c0 con 0x7f09a8129120 2024-09-13T15:42:32.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f09ae0ef640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a809a510 con 0x7f09a8129c60 2024-09-13T15:42:32.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f09ae0ef640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80997f0 con 0x7f09a812a570 2024-09-13T15:42:32.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f09a6ffd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f09a812a570 0x7f09a8128be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f09a6ffd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f09a812a570 0x7f09a8128be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41648/0 (socket says 172.21.15.70:41648) 2024-09-13T15:42:32.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f09a6ffd640 1 -- 172.21.15.70:0/4244434846 learned_addr learned my addr 172.21.15.70:0/4244434846 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:32.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f86f7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f86f8098d20 0x7f86f8093010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38750/0 (socket says 172.21.15.70:38750) 2024-09-13T15:42:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f86f7fff640 1 -- 172.21.15.70:0/3937674104 learned_addr learned my addr 172.21.15.70:0/3937674104 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:42:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f86feb9d640 1 -- 172.21.15.70:0/3937674104 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f806b520 con 0x7f86f8098d20 2024-09-13T15:42:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f86fc939640 1 --2- 172.21.15.70:0/3937674104 >> v2:172.21.15.70:3300/0 conn(0x7f86f80701d0 0x7f86f80987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f86feb9d640 1 -- 172.21.15.70:0/3937674104 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f806c440 con 0x7f86f80701d0 2024-09-13T15:42:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f86feb9d640 1 -- 172.21.15.70:0/3937674104 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8070860 con 0x7f86f8099160 2024-09-13T15:42:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f09a77fe640 1 --2- 172.21.15.70:0/4244434846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09a8129c60 0x7f09a812a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f86fd13a640 1 --2- 172.21.15.70:0/3937674104 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86f8099160 0x7f86f8093720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f09a6ffd640 1 -- 172.21.15.70:0/4244434846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09a8129c60 msgr2=0x7f09a812a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:32.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f09a6ffd640 1 --2- 172.21.15.70:0/4244434846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09a8129c60 0x7f09a812a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f09a6ffd640 1 -- 172.21.15.70:0/4244434846 >> v1:172.21.15.70:6789/0 conn(0x7f09a8129120 legacy=0x7f09a81311d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f09a6ffd640 1 -- 172.21.15.70:0/4244434846 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a8068d10 con 0x7f09a812a570 2024-09-13T15:42:32.139 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f86f77fe640 1 -- 172.21.15.70:0/3937674104 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977268698 0 0) 0x7f86f806b520 con 0x7f86f8098d20 2024-09-13T15:42:32.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f86f77fe640 1 -- 172.21.15.70:0/3937674104 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e80034e0 con 0x7f86f8098d20 2024-09-13T15:42:32.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f86fd13a640 1 -- 172.21.15.70:0/3937674104 >> v2:172.21.15.70:3300/0 conn(0x7f86f80701d0 msgr2=0x7f86f80987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:32.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f86fd13a640 1 --2- 172.21.15.70:0/3937674104 >> v2:172.21.15.70:3300/0 conn(0x7f86f80701d0 0x7f86f80987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f86fd13a640 1 -- 172.21.15.70:0/3937674104 >> v1:172.21.15.70:6789/0 conn(0x7f86f8098d20 legacy=0x7f86f8093010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.134+0000 7f86fd13a640 1 -- 172.21.15.70:0/3937674104 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f806cca0 con 0x7f86f8099160 2024-09-13T15:42:32.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.135+0000 7f09a6ffd640 1 --2- 172.21.15.70:0/4244434846 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f09a812a570 0x7f09a8128be0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f099c009940 tx=0x7f099c0091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fb16c3193a74f07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:32.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.135+0000 7f86fd13a640 1 --2- 172.21.15.70:0/3937674104 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86f8099160 0x7f86f8093720 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009e90 tx=0x7f86f0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=435837b49c02c35a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.135+0000 7f09a67fc640 1 -- 172.21.15.70:0/4244434846 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c02ad00 con 0x7f09a812a570 2024-09-13T15:42:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.135+0000 7f86f77fe640 1 -- 172.21.15.70:0/3937674104 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f002aea0 con 0x7f86f8099160 2024-09-13T15:42:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.135+0000 7f86f77fe640 1 -- 172.21.15.70:0/3937674104 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f002d050 con 0x7f86f8099160 2024-09-13T15:42:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f09a67fc640 1 -- 172.21.15.70:0/4244434846 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c02ae60 con 0x7f09a812a570 2024-09-13T15:42:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f09a67fc640 1 -- 172.21.15.70:0/4244434846 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c032670 con 0x7f09a812a570 2024-09-13T15:42:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.135+0000 7f86f77fe640 1 -- 172.21.15.70:0/3937674104 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0007780 con 0x7f86f8099160 2024-09-13T15:42:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f001f71d640 1 Processor -- start 2024-09-13T15:42:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f001f71d640 1 -- start start 2024-09-13T15:42:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f86feb9d640 1 -- 172.21.15.70:0/3937674104 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86f8099160 msgr2=0x7f86f8093720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f86feb9d640 1 --2- 172.21.15.70:0/3937674104 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86f8099160 0x7f86f8093720 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009e90 tx=0x7f86f0009780 comp rx=0 tx=0).stop 2024-09-13T15:42:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f86feb9d640 1 -- 172.21.15.70:0/3937674104 shutdown_connections 2024-09-13T15:42:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f86feb9d640 1 --2- 172.21.15.70:0/3937674104 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86f8099160 0x7f86f8093720 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f86feb9d640 1 --2- 172.21.15.70:0/3937674104 >> v2:172.21.15.70:3300/0 conn(0x7f86f80701d0 0x7f86f80987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f86feb9d640 1 -- 172.21.15.70:0/3937674104 >> 172.21.15.70:0/3937674104 conn(0x7f86f80963c0 msgr2=0x7f86f80967c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:32.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f09ae0ef640 1 -- 172.21.15.70:0/4244434846 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f09a812a570 msgr2=0x7f09a8128be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f09ae0ef640 1 --2- 172.21.15.70:0/4244434846 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f09a812a570 0x7f09a8128be0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f099c009940 tx=0x7f099c0091d0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f09ae0ef640 1 -- 172.21.15.70:0/4244434846 shutdown_connections 2024-09-13T15:42:32.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f09ae0ef640 1 --2- 172.21.15.70:0/4244434846 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f09a812a570 0x7f09a8128be0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f09ae0ef640 1 --2- 172.21.15.70:0/4244434846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09a8129c60 0x7f09a812a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.144 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f09ae0ef640 1 -- 172.21.15.70:0/4244434846 >> 172.21.15.70:0/4244434846 conn(0x7f09a809de70 msgr2=0x7f09a8068110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:32.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f86feb9d640 1 -- 172.21.15.70:0/3937674104 shutdown_connections 2024-09-13T15:42:32.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f86feb9d640 1 -- 172.21.15.70:0/3937674104 wait complete. 2024-09-13T15:42:32.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f001f71d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00180959a0 0x7f0018095d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.145 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.136+0000 7f09ae0ef640 1 -- 172.21.15.70:0/4244434846 shutdown_connections 2024-09-13T15:42:32.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.137+0000 7f001f71d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f001812a830 0x7f0018132380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.137+0000 7f001f71d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0018063a60 con 0x7f00180962b0 2024-09-13T15:42:32.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.137+0000 7f001f71d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0018062e50 con 0x7f001812a830 2024-09-13T15:42:32.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.137+0000 7f001f71d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001806ebd0 con 0x7f00180959a0 2024-09-13T15:42:32.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.137+0000 7f001d4b9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00180959a0 0x7f0018095d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.137+0000 7f001d4b9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00180959a0 0x7f0018095d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:44338/0 (socket says 172.21.15.70:44338) 2024-09-13T15:42:32.146 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.137+0000 7f001d4b9640 1 -- 172.21.15.70:0/153254306 learned_addr learned my addr 172.21.15.70:0/153254306 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:32.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.137+0000 7f001dcba640 1 --2- 172.21.15.70:0/153254306 >> v2:172.21.15.70:3300/0 conn(0x7f001812a830 0x7f0018132380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.137+0000 7f001d4b9640 1 -- 172.21.15.70:0/153254306 >> v2:172.21.15.70:3300/0 conn(0x7f001812a830 msgr2=0x7f0018132380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:32.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.137+0000 7f001d4b9640 1 --2- 172.21.15.70:0/153254306 >> v2:172.21.15.70:3300/0 conn(0x7f001812a830 0x7f0018132380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.137+0000 7f001d4b9640 1 -- 172.21.15.70:0/153254306 >> v1:172.21.15.70:6789/0 conn(0x7f00180962b0 legacy=0x7f001812a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.137+0000 7f001d4b9640 1 -- 172.21.15.70:0/153254306 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001806f240 con 0x7f00180959a0 2024-09-13T15:42:32.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.137+0000 7f09ae0ef640 1 -- 172.21.15.70:0/4244434846 wait complete. 2024-09-13T15:42:32.147 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.137+0000 7f86feb9d640 1 Processor -- start 2024-09-13T15:42:32.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f86feb9d640 1 -- start start 2024-09-13T15:42:32.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f001d4b9640 1 --2- 172.21.15.70:0/153254306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00180959a0 0x7f0018095d70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0014009910 tx=0x7f00140091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=939934601018a6b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:32.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f000ffff640 1 -- 172.21.15.70:0/153254306 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001402ad20 con 0x7f00180959a0 2024-09-13T15:42:32.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f86feb9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86f80701d0 0x7f86f813d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.148 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f86feb9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86f8098d20 0x7f86f813dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f000ffff640 1 -- 172.21.15.70:0/153254306 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001402ae80 con 0x7f00180959a0 2024-09-13T15:42:32.149 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f000ffff640 1 -- 172.21.15.70:0/153254306 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0014032670 con 0x7f00180959a0 2024-09-13T15:42:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f86feb9d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86f8099160 0x7f86f8087f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86f8098d20 0x7f86f813dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f86f7fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86f8098d20 0x7f86f813dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:44352/0 (socket says 172.21.15.70:44352) 2024-09-13T15:42:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f86f7fff640 1 -- 172.21.15.70:0/3083496896 learned_addr learned my addr 172.21.15.70:0/3083496896 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f86feb9d640 1 -- 172.21.15.70:0/3083496896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f808b3b0 con 0x7f86f80701d0 2024-09-13T15:42:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f86feb9d640 1 -- 172.21.15.70:0/3083496896 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80896b0 con 0x7f86f8098d20 2024-09-13T15:42:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f86feb9d640 1 -- 172.21.15.70:0/3083496896 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86f808b030 con 0x7f86f8099160 2024-09-13T15:42:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f86fd13a640 1 --2- 172.21.15.70:0/3083496896 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86f8099160 0x7f86f8087f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f09ae0ef640 1 Processor -- start 2024-09-13T15:42:32.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f86fc939640 1 --2- 172.21.15.70:0/3083496896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86f80701d0 0x7f86f813d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.138+0000 7f09ae0ef640 1 -- start start 2024-09-13T15:42:32.151 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f86f7fff640 1 -- 172.21.15.70:0/3083496896 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86f8099160 msgr2=0x7f86f8087f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f86f7fff640 1 --2- 172.21.15.70:0/3083496896 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86f8099160 0x7f86f8087f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f86f7fff640 1 -- 172.21.15.70:0/3083496896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86f80701d0 msgr2=0x7f86f813d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f86f7fff640 1 --2- 172.21.15.70:0/3083496896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86f80701d0 0x7f86f813d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f86f7fff640 1 -- 172.21.15.70:0/3083496896 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f0009a20 con 0x7f86f8098d20 2024-09-13T15:42:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f86fd13a640 1 --2- 172.21.15.70:0/3083496896 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86f8099160 0x7f86f8087f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f86f7fff640 1 --2- 172.21.15.70:0/3083496896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86f8098d20 0x7f86f813dab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f86e400c5b0 tx=0x7f86e4007d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f09ae0ef640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a8129120 0x7f09a81392d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f001f71d640 1 -- 172.21.15.70:0/153254306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00180959a0 msgr2=0x7f0018095d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f001f71d640 1 --2- 172.21.15.70:0/153254306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00180959a0 0x7f0018095d70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0014009910 tx=0x7f00140091d0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f09ae0ef640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09a8129c60 0x7f09a8139810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.153 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f86f4ff9640 1 -- 172.21.15.70:0/3083496896 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4005aa0 con 0x7f86f8098d20 2024-09-13T15:42:32.154 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f09ae0ef640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f09a812a570 0x7f09a8139d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.154 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f86feb9d640 1 -- 172.21.15.70:0/3083496896 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f808a870 con 0x7f86f8098d20 2024-09-13T15:42:32.154 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f86f4ff9640 1 -- 172.21.15.70:0/3083496896 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e4005c00 con 0x7f86f8098d20 2024-09-13T15:42:32.154 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f86f4ff9640 1 -- 172.21.15.70:0/3083496896 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40070f0 con 0x7f86f8098d20 2024-09-13T15:42:32.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f09ae0ef640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a811c3d0 con 0x7f09a8129120 2024-09-13T15:42:32.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f001f71d640 1 -- 172.21.15.70:0/153254306 shutdown_connections 2024-09-13T15:42:32.155 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f09ae0ef640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a811c7a0 con 0x7f09a8129c60 2024-09-13T15:42:32.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f001f71d640 1 --2- 172.21.15.70:0/153254306 >> v2:172.21.15.70:3300/0 conn(0x7f001812a830 0x7f0018132380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f001f71d640 1 --2- 172.21.15.70:0/153254306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f00180959a0 0x7f0018095d70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.156 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f001f71d640 1 -- 172.21.15.70:0/153254306 >> 172.21.15.70:0/153254306 conn(0x7f0018096a90 msgr2=0x7f0018096e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:32.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f09ae0ef640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f09a811aaa0 con 0x7f09a812a570 2024-09-13T15:42:32.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f09a7fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f09a812a570 0x7f09a8139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f09a7fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f09a812a570 0x7f09a8139d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41676/0 (socket says 172.21.15.70:41676) 2024-09-13T15:42:32.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f09a7fff640 1 -- 172.21.15.70:0/29155116 learned_addr learned my addr 172.21.15.70:0/29155116 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:32.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f09a77fe640 1 --2- 172.21.15.70:0/29155116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a8129120 0x7f09a81392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.140+0000 7f09a6ffd640 1 --2- 172.21.15.70:0/29155116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09a8129c60 0x7f09a8139810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.140+0000 7f09a7fff640 1 -- 172.21.15.70:0/29155116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09a8129c60 msgr2=0x7f09a8139810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.140+0000 7f09a7fff640 1 --2- 172.21.15.70:0/29155116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09a8129c60 0x7f09a8139810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.140+0000 7f09a7fff640 1 -- 172.21.15.70:0/29155116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a8129120 msgr2=0x7f09a81392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.140+0000 7f09a7fff640 1 --2- 172.21.15.70:0/29155116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a8129120 0x7f09a81392d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.157 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.140+0000 7f09a7fff640 1 -- 172.21.15.70:0/29155116 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c009450 con 0x7f09a812a570 2024-09-13T15:42:32.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.140+0000 7f09a77fe640 1 --2- 172.21.15.70:0/29155116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a8129120 0x7f09a81392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.139+0000 7f86feb9d640 1 -- 172.21.15.70:0/3083496896 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f8089de0 con 0x7f86f8098d20 2024-09-13T15:42:32.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.140+0000 7f001f71d640 1 -- 172.21.15.70:0/153254306 shutdown_connections 2024-09-13T15:42:32.158 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.140+0000 7f09a7fff640 1 --2- 172.21.15.70:0/29155116 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f09a812a570 0x7f09a8139d50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f09a000c860 tx=0x7f09a0001690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:32.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.140+0000 7f001f71d640 1 -- 172.21.15.70:0/153254306 wait complete. 2024-09-13T15:42:32.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.140+0000 7f098f7fe640 1 -- 172.21.15.70:0/29155116 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a000ce50 con 0x7f09a812a570 2024-09-13T15:42:32.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.140+0000 7f098f7fe640 1 -- 172.21.15.70:0/29155116 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a000a850 con 0x7f09a812a570 2024-09-13T15:42:32.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.140+0000 7f098f7fe640 1 -- 172.21.15.70:0/29155116 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a000ac10 con 0x7f09a812a570 2024-09-13T15:42:32.159 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.141+0000 7f09ae0ef640 1 -- 172.21.15.70:0/29155116 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a811bbf0 con 0x7f09a812a570 2024-09-13T15:42:32.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.140+0000 7f86feb9d640 1 -- 172.21.15.70:0/3083496896 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f805ed70 con 0x7f86f8098d20 2024-09-13T15:42:32.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.141+0000 7f09ae0ef640 1 -- 172.21.15.70:0/29155116 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a81191b0 con 0x7f09a812a570 2024-09-13T15:42:32.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.141+0000 7f86f4ff9640 1 -- 172.21.15.70:0/3083496896 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f86e4007520 con 0x7f86f8098d20 2024-09-13T15:42:32.160 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.141+0000 7f86f4ff9640 1 -- 172.21.15.70:0/3083496896 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f86e4062de0 con 0x7f86f8098d20 2024-09-13T15:42:32.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.142+0000 7f001f71d640 1 Processor -- start 2024-09-13T15:42:32.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.142+0000 7f001f71d640 1 -- start start 2024-09-13T15:42:32.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.142+0000 7f098f7fe640 1 -- 172.21.15.70:0/29155116 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f09a0003030 con 0x7f09a812a570 2024-09-13T15:42:32.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.142+0000 7f09ae0ef640 1 -- 172.21.15.70:0/29155116 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a805ed70 con 0x7f09a812a570 2024-09-13T15:42:32.161 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.142+0000 7f098f7fe640 1 -- 172.21.15.70:0/29155116 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f09a0066d20 con 0x7f09a812a570 2024-09-13T15:42:32.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.142+0000 7f001f71d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f00180959a0 0x7f00181393b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.142+0000 7f001f71d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00180962b0 0x7f00181398f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.142+0000 7f001f71d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f001812a830 0x7f0018139e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001f71d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001808d670 con 0x7f00180962b0 2024-09-13T15:42:32.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001f71d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001812a2e0 con 0x7f001812a830 2024-09-13T15:42:32.162 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001f71d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f001808d8a0 con 0x7f00180959a0 2024-09-13T15:42:32.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001d4b9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f00180959a0 0x7f00181393b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001d4b9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f00180959a0 0x7f00181393b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41686/0 (socket says 172.21.15.70:41686) 2024-09-13T15:42:32.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001d4b9640 1 -- 172.21.15.70:0/1566104602 learned_addr learned my addr 172.21.15.70:0/1566104602 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:32.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001dcba640 1 --2- 172.21.15.70:0/1566104602 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f001812a830 0x7f0018139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001ccb8640 1 --2- 172.21.15.70:0/1566104602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00180962b0 0x7f00181398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001d4b9640 1 -- 172.21.15.70:0/1566104602 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f001812a830 msgr2=0x7f0018139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001d4b9640 1 --2- 172.21.15.70:0/1566104602 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f001812a830 0x7f0018139e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001d4b9640 1 -- 172.21.15.70:0/1566104602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00180962b0 msgr2=0x7f00181398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001d4b9640 1 --2- 172.21.15.70:0/1566104602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00180962b0 0x7f00181398f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001d4b9640 1 -- 172.21.15.70:0/1566104602 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0014009450 con 0x7f00180959a0 2024-09-13T15:42:32.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001dcba640 1 --2- 172.21.15.70:0/1566104602 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f001812a830 0x7f0018139e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.163 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001ccb8640 1 --2- 172.21.15.70:0/1566104602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00180962b0 0x7f00181398f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:42:32.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.143+0000 7f001d4b9640 1 --2- 172.21.15.70:0/1566104602 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f00180959a0 0x7f00181393b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0014032d90 tx=0x7f0014032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:32.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.145+0000 7f86f4ff9640 1 -- 172.21.15.70:0/3083496896 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f86e403df60 con 0x7f86f8098d20 2024-09-13T15:42:32.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.147+0000 7f000d7fa640 1 -- 172.21.15.70:0/1566104602 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001403c620 con 0x7f00180959a0 2024-09-13T15:42:32.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.147+0000 7f000d7fa640 1 -- 172.21.15.70:0/1566104602 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001403c780 con 0x7f00180959a0 2024-09-13T15:42:32.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.147+0000 7f000d7fa640 1 -- 172.21.15.70:0/1566104602 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001403c620 con 0x7f00180959a0 2024-09-13T15:42:32.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.147+0000 7f001f71d640 1 -- 172.21.15.70:0/1566104602 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001808bba0 con 0x7f00180959a0 2024-09-13T15:42:32.165 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.147+0000 7f098f7fe640 1 -- 172.21.15.70:0/29155116 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f09a0068060 con 0x7f09a812a570 2024-09-13T15:42:32.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.147+0000 7f001f71d640 1 -- 172.21.15.70:0/1566104602 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001808a2e0 con 0x7f00180959a0 2024-09-13T15:42:32.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.148+0000 7f001f71d640 1 -- 172.21.15.70:0/1566104602 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001808dac0 con 0x7f00180959a0 2024-09-13T15:42:32.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.148+0000 7f000d7fa640 1 -- 172.21.15.70:0/1566104602 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f0014039070 con 0x7f00180959a0 2024-09-13T15:42:32.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.149+0000 7f000d7fa640 1 -- 172.21.15.70:0/1566104602 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f0014082be0 con 0x7f00180959a0 2024-09-13T15:42:32.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.155+0000 7f000d7fa640 1 -- 172.21.15.70:0/1566104602 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f0014061b30 con 0x7f00180959a0 2024-09-13T15:42:32.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.252+0000 7f627b15d640 1 -- 172.21.15.70:0/927758294 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f627405ed70 con 0x7f6274096bc0 2024-09-13T15:42:32.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.253+0000 7f6270ff9640 1 -- 172.21.15.70:0/927758294 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6260042070 con 0x7f6274096bc0 2024-09-13T15:42:32.254 INFO:teuthology.orchestra.run.smithi070.stdout:38654705669 2024-09-13T15:42:32.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.255+0000 7f627b15d640 1 -- 172.21.15.70:0/927758294 >> v1:172.21.15.70:6816/32242 conn(0x7f625804cb70 legacy=0x7f625804f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.255+0000 7f627b15d640 1 -- 172.21.15.70:0/927758294 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6274096bc0 msgr2=0x7f62740884a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.256 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.255+0000 7f627b15d640 1 --2- 172.21.15.70:0/927758294 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6274096bc0 0x7f62740884a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f626002a4f0 tx=0x7f6260032000 comp rx=0 tx=0).stop 2024-09-13T15:42:32.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.255+0000 7f627b15d640 1 -- 172.21.15.70:0/927758294 shutdown_connections 2024-09-13T15:42:32.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.255+0000 7f627b15d640 1 --2- 172.21.15.70:0/927758294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f627412c330 0x7f62740889e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.255+0000 7f627b15d640 1 --2- 172.21.15.70:0/927758294 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6274096bc0 0x7f62740884a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.255+0000 7f627b15d640 1 --2- 172.21.15.70:0/927758294 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f627408f820 0x7f6274087f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.255+0000 7f627b15d640 1 -- 172.21.15.70:0/927758294 >> 172.21.15.70:0/927758294 conn(0x7f6274097cb0 msgr2=0x7f627412b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:32.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.255+0000 7f627b15d640 1 -- 172.21.15.70:0/927758294 shutdown_connections 2024-09-13T15:42:32.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.255+0000 7f627b15d640 1 -- 172.21.15.70:0/927758294 wait complete. 2024-09-13T15:42:32.267 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705669 for osd.2 2024-09-13T15:42:32.267 DEBUG:teuthology.parallel:result is None 2024-09-13T15:42:32.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.284+0000 7f09ae0ef640 1 -- 172.21.15.70:0/29155116 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f09a8070480 con 0x7f09a812a570 2024-09-13T15:42:32.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.284+0000 7f098f7fe640 1 -- 172.21.15.70:0/29155116 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f09a003d990 con 0x7f09a812a570 2024-09-13T15:42:32.286 INFO:teuthology.orchestra.run.smithi070.stdout:38654705669 2024-09-13T15:42:32.287 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.286+0000 7f09ae0ef640 1 -- 172.21.15.70:0/29155116 >> v1:172.21.15.70:6816/32242 conn(0x7f099404cb70 legacy=0x7f099404f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.286+0000 7f09ae0ef640 1 -- 172.21.15.70:0/29155116 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f09a812a570 msgr2=0x7f09a8139d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.286+0000 7f09ae0ef640 1 --2- 172.21.15.70:0/29155116 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f09a812a570 0x7f09a8139d50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f09a000c860 tx=0x7f09a0001690 comp rx=0 tx=0).stop 2024-09-13T15:42:32.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.286+0000 7f09ae0ef640 1 -- 172.21.15.70:0/29155116 shutdown_connections 2024-09-13T15:42:32.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.286+0000 7f09ae0ef640 1 --2- 172.21.15.70:0/29155116 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f09a812a570 0x7f09a8139d50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.286+0000 7f09ae0ef640 1 --2- 172.21.15.70:0/29155116 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f09a8129c60 0x7f09a8139810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.286+0000 7f09ae0ef640 1 --2- 172.21.15.70:0/29155116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a8129120 0x7f09a81392d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.286+0000 7f09ae0ef640 1 -- 172.21.15.70:0/29155116 >> 172.21.15.70:0/29155116 conn(0x7f09a809de70 msgr2=0x7f09a812cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:32.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.286+0000 7f09ae0ef640 1 -- 172.21.15.70:0/29155116 shutdown_connections 2024-09-13T15:42:32.288 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.286+0000 7f09ae0ef640 1 -- 172.21.15.70:0/29155116 wait complete. 2024-09-13T15:42:32.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.288+0000 7f86feb9d640 1 -- 172.21.15.70:0/3083496896 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f86f8068330 con 0x7f86f8098d20 2024-09-13T15:42:32.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.289+0000 7f86f4ff9640 1 -- 172.21.15.70:0/3083496896 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f86e4041770 con 0x7f86f8098d20 2024-09-13T15:42:32.290 INFO:teuthology.orchestra.run.smithi070.stdout:38654705668 2024-09-13T15:42:32.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.291+0000 7f86feb9d640 1 -- 172.21.15.70:0/3083496896 >> v1:172.21.15.70:6816/32242 conn(0x7f86dc04cb70 legacy=0x7f86dc04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.291+0000 7f86feb9d640 1 -- 172.21.15.70:0/3083496896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86f8098d20 msgr2=0x7f86f813dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.291+0000 7f86feb9d640 1 --2- 172.21.15.70:0/3083496896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86f8098d20 0x7f86f813dab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f86e400c5b0 tx=0x7f86e4007d70 comp rx=0 tx=0).stop 2024-09-13T15:42:32.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.291+0000 7f86feb9d640 1 -- 172.21.15.70:0/3083496896 shutdown_connections 2024-09-13T15:42:32.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.291+0000 7f86feb9d640 1 --2- 172.21.15.70:0/3083496896 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f86f8099160 0x7f86f8087f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.291+0000 7f86feb9d640 1 --2- 172.21.15.70:0/3083496896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86f8098d20 0x7f86f813dab0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.291+0000 7f86feb9d640 1 --2- 172.21.15.70:0/3083496896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86f80701d0 0x7f86f813d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.291+0000 7f86feb9d640 1 -- 172.21.15.70:0/3083496896 >> 172.21.15.70:0/3083496896 conn(0x7f86f80963c0 msgr2=0x7f86f812d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:32.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.291+0000 7f86feb9d640 1 -- 172.21.15.70:0/3083496896 shutdown_connections 2024-09-13T15:42:32.293 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.291+0000 7f86feb9d640 1 -- 172.21.15.70:0/3083496896 wait complete. 2024-09-13T15:42:32.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.294+0000 7f001f71d640 1 -- 172.21.15.70:0/1566104602 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f001805ed70 con 0x7f00180959a0 2024-09-13T15:42:32.295 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.294+0000 7f000d7fa640 1 -- 172.21.15.70:0/1566104602 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0014037070 con 0x7f00180959a0 2024-09-13T15:42:32.295 INFO:teuthology.orchestra.run.smithi070.stdout:38654705669 2024-09-13T15:42:32.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.296+0000 7f001f71d640 1 -- 172.21.15.70:0/1566104602 >> v1:172.21.15.70:6816/32242 conn(0x7efffc04cc30 legacy=0x7efffc04f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.296+0000 7f001f71d640 1 -- 172.21.15.70:0/1566104602 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f00180959a0 msgr2=0x7f00181393b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.296+0000 7f001f71d640 1 --2- 172.21.15.70:0/1566104602 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f00180959a0 0x7f00181393b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0014032d90 tx=0x7f0014032000 comp rx=0 tx=0).stop 2024-09-13T15:42:32.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.296+0000 7f001f71d640 1 -- 172.21.15.70:0/1566104602 shutdown_connections 2024-09-13T15:42:32.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.296+0000 7f001f71d640 1 --2- 172.21.15.70:0/1566104602 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f001812a830 0x7f0018139e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.296+0000 7f001f71d640 1 --2- 172.21.15.70:0/1566104602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00180962b0 0x7f00181398f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.296+0000 7f001f71d640 1 --2- 172.21.15.70:0/1566104602 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f00180959a0 0x7f00181393b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.296+0000 7f001f71d640 1 -- 172.21.15.70:0/1566104602 >> 172.21.15.70:0/1566104602 conn(0x7f0018096a90 msgr2=0x7f001808f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:32.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.296+0000 7f001f71d640 1 -- 172.21.15.70:0/1566104602 shutdown_connections 2024-09-13T15:42:32.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.296+0000 7f001f71d640 1 -- 172.21.15.70:0/1566104602 wait complete. 2024-09-13T15:42:32.300 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705669 for osd.3 2024-09-13T15:42:32.300 DEBUG:teuthology.parallel:result is None 2024-09-13T15:42:32.304 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705668 for osd.1 2024-09-13T15:42:32.305 DEBUG:teuthology.parallel:result is None 2024-09-13T15:42:32.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705669 for osd.0 2024-09-13T15:42:32.307 DEBUG:teuthology.parallel:result is None 2024-09-13T15:42:32.307 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T15:42:32.308 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T15:42:32.441 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.439+0000 7f0ef239d640 1 Processor -- start 2024-09-13T15:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.439+0000 7f0ef239d640 1 -- start start 2024-09-13T15:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.440+0000 7f0ef239d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0eec090a40 0x7f0eec090e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.440+0000 7f0ef239d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eec126cc0 0x7f0eec12e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.440+0000 7f0ef239d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eec090480 con 0x7f0eec1261c0 2024-09-13T15:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.440+0000 7f0ef239d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec092ac0 con 0x7f0eec126cc0 2024-09-13T15:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.440+0000 7f0ef239d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec092d10 con 0x7f0eec090a40 2024-09-13T15:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.440+0000 7f0eebfff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0eec090a40 0x7f0eec090e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.440+0000 7f0ef093a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eec126cc0 0x7f0eec12e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.440+0000 7f0eeb7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0eec1261c0 0x7f0eec1265b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38788/0 (socket says 172.21.15.70:38788) 2024-09-13T15:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.440+0000 7f0eeb7fe640 1 -- 172.21.15.70:0/4259791849 learned_addr learned my addr 172.21.15.70:0/4259791849 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.440+0000 7f0eeaffd640 1 -- 172.21.15.70:0/4259791849 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967525138 0 0) 0x7f0eec090480 con 0x7f0eec1261c0 2024-09-13T15:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.440+0000 7f0ef093a640 1 -- 172.21.15.70:0/4259791849 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0eec090a40 msgr2=0x7f0eec090e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:32.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.440+0000 7f0ef093a640 1 --2- 172.21.15.70:0/4259791849 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0eec090a40 0x7f0eec090e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.440+0000 7f0ef093a640 1 -- 172.21.15.70:0/4259791849 >> v1:172.21.15.70:6789/0 conn(0x7f0eec1261c0 legacy=0x7f0eec1265b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.440+0000 7f0ef093a640 1 -- 172.21.15.70:0/4259791849 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec0953c0 con 0x7f0eec126cc0 2024-09-13T15:42:32.443 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.440+0000 7f0eebfff640 1 --2- 172.21.15.70:0/4259791849 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0eec090a40 0x7f0eec090e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.441+0000 7f0ef093a640 1 --2- 172.21.15.70:0/4259791849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eec126cc0 0x7f0eec12e510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009e90 tx=0x7f0ee4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=617daba7fd2c42cf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:32.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.441+0000 7f0eeaffd640 1 -- 172.21.15.70:0/4259791849 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4031400 con 0x7f0eec126cc0 2024-09-13T15:42:32.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.441+0000 7f0eeaffd640 1 -- 172.21.15.70:0/4259791849 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee402d050 con 0x7f0eec126cc0 2024-09-13T15:42:32.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.441+0000 7f0eeaffd640 1 -- 172.21.15.70:0/4259791849 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4031790 con 0x7f0eec126cc0 2024-09-13T15:42:32.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.441+0000 7f0ef239d640 1 -- 172.21.15.70:0/4259791849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eec126cc0 msgr2=0x7f0eec12e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.441+0000 7f0ef239d640 1 --2- 172.21.15.70:0/4259791849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eec126cc0 0x7f0eec12e510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009e90 tx=0x7f0ee4009780 comp rx=0 tx=0).stop 2024-09-13T15:42:32.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.442+0000 7f0ef239d640 1 -- 172.21.15.70:0/4259791849 shutdown_connections 2024-09-13T15:42:32.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.442+0000 7f0ef239d640 1 --2- 172.21.15.70:0/4259791849 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eec126cc0 0x7f0eec12e510 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.442+0000 7f0ef239d640 1 --2- 172.21.15.70:0/4259791849 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0eec090a40 0x7f0eec090e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.442+0000 7f0ef239d640 1 -- 172.21.15.70:0/4259791849 >> 172.21.15.70:0/4259791849 conn(0x7f0eec0959f0 msgr2=0x7f0eec095df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:32.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.442+0000 7f0ef239d640 1 -- 172.21.15.70:0/4259791849 shutdown_connections 2024-09-13T15:42:32.450 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.442+0000 7f0ef239d640 1 -- 172.21.15.70:0/4259791849 wait complete. 2024-09-13T15:42:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.442+0000 7f0ef239d640 1 Processor -- start 2024-09-13T15:42:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.442+0000 7f0ef239d640 1 -- start start 2024-09-13T15:42:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0ef239d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0eec090a40 0x7f0eec1218d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0ef239d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eec1261c0 0x7f0eec121e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0ef239d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0eec126cc0 0x7f0eec122350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0ef239d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1238e0 con 0x7f0eec126cc0 2024-09-13T15:42:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0ef239d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1252d0 con 0x7f0eec1261c0 2024-09-13T15:42:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0ef239d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec123360 con 0x7f0eec090a40 2024-09-13T15:42:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0eebfff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0eec090a40 0x7f0eec1218d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0ef093a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0eec126cc0 0x7f0eec122350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0eeb7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eec1261c0 0x7f0eec121e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0eebfff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0eec090a40 0x7f0eec1218d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41698/0 (socket says 172.21.15.70:41698) 2024-09-13T15:42:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0eebfff640 1 -- 172.21.15.70:0/982980964 learned_addr learned my addr 172.21.15.70:0/982980964 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0eebfff640 1 -- 172.21.15.70:0/982980964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eec1261c0 msgr2=0x7f0eec121e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0eebfff640 1 --2- 172.21.15.70:0/982980964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eec1261c0 0x7f0eec121e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0eebfff640 1 -- 172.21.15.70:0/982980964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0eec126cc0 msgr2=0x7f0eec122350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0eebfff640 1 --2- 172.21.15.70:0/982980964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0eec126cc0 0x7f0eec122350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0eebfff640 1 -- 172.21.15.70:0/982980964 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc00a530 con 0x7f0eec090a40 2024-09-13T15:42:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0eeb7fe640 1 --2- 172.21.15.70:0/982980964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eec1261c0 0x7f0eec121e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0ef093a640 1 --2- 172.21.15.70:0/982980964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0eec126cc0 0x7f0eec122350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.443+0000 7f0eebfff640 1 --2- 172.21.15.70:0/982980964 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0eec090a40 0x7f0eec1218d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00fbb0 tx=0x7f0edc00a2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.444+0000 7f0ed3fff640 1 -- 172.21.15.70:0/982980964 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc00dc40 con 0x7f0eec090a40 2024-09-13T15:42:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.444+0000 7f0ed3fff640 1 -- 172.21.15.70:0/982980964 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0edc00dda0 con 0x7f0eec090a40 2024-09-13T15:42:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.444+0000 7f0ef239d640 1 -- 172.21.15.70:0/982980964 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee4009a20 con 0x7f0eec090a40 2024-09-13T15:42:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.444+0000 7f0ed3fff640 1 -- 172.21.15.70:0/982980964 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc01b6b0 con 0x7f0eec090a40 2024-09-13T15:42:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.444+0000 7f0ef239d640 1 -- 172.21.15.70:0/982980964 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eec122890 con 0x7f0eec090a40 2024-09-13T15:42:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.445+0000 7f0ef239d640 1 -- 172.21.15.70:0/982980964 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eec124390 con 0x7f0eec090a40 2024-09-13T15:42:32.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.445+0000 7f0ed3fff640 1 -- 172.21.15.70:0/982980964 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f0edc01ba60 con 0x7f0eec090a40 2024-09-13T15:42:32.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.445+0000 7f0ed3fff640 1 -- 172.21.15.70:0/982980964 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f0edc016070 con 0x7f0eec090a40 2024-09-13T15:42:32.453 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.448+0000 7f0ed3fff640 1 -- 172.21.15.70:0/982980964 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f0edc03fc70 con 0x7f0eec090a40 2024-09-13T15:42:32.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.576+0000 7f0ef239d640 1 -- 172.21.15.70:0/982980964 --> v1:172.21.15.70:6816/32242 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0eec05ed70 con 0x7f0ed804cb70 2024-09-13T15:42:32.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.577+0000 7f0ed3fff640 1 -- 172.21.15.70:0/982980964 <== mgr.4104 v1:172.21.15.70:6816/32242 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34969 (unknown 2965378022 0 3392297555) 0x7f0eec05ed70 con 0x7f0ed804cb70 2024-09-13T15:42:32.579 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:42:32.579 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-13T15:42:32.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.579+0000 7f0ef239d640 1 -- 172.21.15.70:0/982980964 >> v1:172.21.15.70:6816/32242 conn(0x7f0ed804cb70 legacy=0x7f0ed804f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.579+0000 7f0ef239d640 1 -- 172.21.15.70:0/982980964 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0eec090a40 msgr2=0x7f0eec1218d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.579+0000 7f0ef239d640 1 --2- 172.21.15.70:0/982980964 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0eec090a40 0x7f0eec1218d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00fbb0 tx=0x7f0edc00a2b0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.579+0000 7f0ef239d640 1 -- 172.21.15.70:0/982980964 shutdown_connections 2024-09-13T15:42:32.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.579+0000 7f0ef239d640 1 --2- 172.21.15.70:0/982980964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0eec126cc0 0x7f0eec122350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.579+0000 7f0ef239d640 1 --2- 172.21.15.70:0/982980964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0eec1261c0 0x7f0eec121e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.579+0000 7f0ef239d640 1 --2- 172.21.15.70:0/982980964 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0eec090a40 0x7f0eec1218d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.579+0000 7f0ef239d640 1 -- 172.21.15.70:0/982980964 >> 172.21.15.70:0/982980964 conn(0x7f0eec0959f0 msgr2=0x7f0eec0978a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.579+0000 7f0ef239d640 1 -- 172.21.15.70:0/982980964 shutdown_connections 2024-09-13T15:42:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.579+0000 7f0ef239d640 1 -- 172.21.15.70:0/982980964 wait complete. 2024-09-13T15:42:32.590 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-13T15:42:30.971937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1207784,"kb_used_data":1712,"kb_used_omap":6,"kb_used_meta":1206009,"kb_avail":376279576,"statfs":{"total":386547056640,"available":385310285824,"internally_reserved":0,"allocated":1753088,"data_stored":1335854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1234953568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.262342"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.720294+0000","last_change":"2024-09-13T15:42:29.720527+0000","last_active":"2024-09-13T15:42:29.720294+0000","last_peered":"2024-09-13T15:42:29.720294+0000","last_clean":"2024-09-13T15:42:29.720294+0000","last_became_active":"2024-09-13T15:42:27.716998+0000","last_became_peered":"2024-09-13T15:42:27.716998+0000","last_unstale":"2024-09-13T15:42:29.720294+0000","last_undegraded":"2024-09-13T15:42:29.720294+0000","last_fullsized":"2024-09-13T15:42:29.720294+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:22.475936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035985100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.717506+0000","last_change":"2024-09-13T15:42:29.717707+0000","last_active":"2024-09-13T15:42:29.717506+0000","last_peered":"2024-09-13T15:42:29.717506+0000","last_clean":"2024-09-13T15:42:29.717506+0000","last_became_active":"2024-09-13T15:42:27.718860+0000","last_became_peered":"2024-09-13T15:42:27.718860+0000","last_unstale":"2024-09-13T15:42:29.717506+0000","last_undegraded":"2024-09-13T15:42:29.717506+0000","last_fullsized":"2024-09-13T15:42:29.717506+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:19.555928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054553999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.721176+0000","last_change":"2024-09-13T15:42:29.721358+0000","last_active":"2024-09-13T15:42:29.721176+0000","last_peered":"2024-09-13T15:42:29.721176+0000","last_clean":"2024-09-13T15:42:29.721176+0000","last_became_active":"2024-09-13T15:42:27.717168+0000","last_became_peered":"2024-09-13T15:42:27.717168+0000","last_unstale":"2024-09-13T15:42:29.721176+0000","last_undegraded":"2024-09-13T15:42:29.721176+0000","last_fullsized":"2024-09-13T15:42:29.721176+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:15.334763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011766770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.717555+0000","last_change":"2024-09-13T15:42:29.717708+0000","last_active":"2024-09-13T15:42:29.717555+0000","last_peered":"2024-09-13T15:42:29.717555+0000","last_clean":"2024-09-13T15:42:29.717555+0000","last_became_active":"2024-09-13T15:42:27.719566+0000","last_became_peered":"2024-09-13T15:42:27.719566+0000","last_unstale":"2024-09-13T15:42:29.717555+0000","last_undegraded":"2024-09-13T15:42:29.717555+0000","last_fullsized":"2024-09-13T15:42:29.717555+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:59.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024452499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"16'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.721120+0000","last_change":"2024-09-13T15:42:29.721120+0000","last_active":"2024-09-13T15:42:29.721120+0000","last_peered":"2024-09-13T15:42:29.721120+0000","last_clean":"2024-09-13T15:42:29.721120+0000","last_became_active":"2024-09-13T15:42:27.718029+0000","last_became_peered":"2024-09-13T15:42:27.718029+0000","last_unstale":"2024-09-13T15:42:29.721120+0000","last_undegraded":"2024-09-13T15:42:29.721120+0000","last_fullsized":"2024-09-13T15:42:29.721120+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:44.778809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094542300000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.717199+0000","last_change":"2024-09-13T15:42:29.717606+0000","last_active":"2024-09-13T15:42:29.717199+0000","last_peered":"2024-09-13T15:42:29.717199+0000","last_clean":"2024-09-13T15:42:29.717199+0000","last_became_active":"2024-09-13T15:42:27.715108+0000","last_became_peered":"2024-09-13T15:42:27.715108+0000","last_unstale":"2024-09-13T15:42:29.717199+0000","last_undegraded":"2024-09-13T15:42:29.717199+0000","last_fullsized":"2024-09-13T15:42:29.717199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:34.509753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089033199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.720264+0000","last_change":"2024-09-13T15:42:29.720489+0000","last_active":"2024-09-13T15:42:29.720264+0000","last_peered":"2024-09-13T15:42:29.720264+0000","last_clean":"2024-09-13T15:42:29.720264+0000","last_became_active":"2024-09-13T15:42:27.718347+0000","last_became_peered":"2024-09-13T15:42:27.718347+0000","last_unstale":"2024-09-13T15:42:29.720264+0000","last_undegraded":"2024-09-13T15:42:29.720264+0000","last_fullsized":"2024-09-13T15:42:29.720264+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:16.064731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034895899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"14'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.720963+0000","last_change":"2024-09-13T15:42:29.721205+0000","last_active":"2024-09-13T15:42:29.720963+0000","last_peered":"2024-09-13T15:42:29.720963+0000","last_clean":"2024-09-13T15:42:29.720963+0000","last_became_active":"2024-09-13T15:42:27.720033+0000","last_became_peered":"2024-09-13T15:42:27.720033+0000","last_unstale":"2024-09-13T15:42:29.720963+0000","last_undegraded":"2024-09-13T15:42:29.720963+0000","last_fullsized":"2024-09-13T15:42:29.720963+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:36.892284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276560000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"11'52","reported_seq":80,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T15:42:28.713720+0000","last_change":"2024-09-13T15:42:22.700543+0000","last_active":"2024-09-13T15:42:28.713720+0000","last_peered":"2024-09-13T15:42:28.713720+0000","last_clean":"2024-09-13T15:42:28.713720+0000","last_became_active":"2024-09-13T15:42:22.700226+0000","last_became_peered":"2024-09-13T15:42:22.700226+0000","last_unstale":"2024-09-13T15:42:28.713720+0000","last_undegraded":"2024-09-13T15:42:28.713720+0000","last_fullsized":"2024-09-13T15:42:28.713720+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_clean_scrub_stamp":"2024-09-13T15:42:21.689345+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:02.263603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301660,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070180,"statfs":{"total":96636764160,"available":96327864320,"internally_reserved":0,"allocated":143360,"data_stored":38789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]}]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301656,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070184,"statfs":{"total":96636764160,"available":96327868416,"internally_reserved":0,"allocated":139264,"data_stored":38770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302236,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069604,"statfs":{"total":96636764160,"available":96327274496,"internally_reserved":0,"allocated":737280,"data_stored":629157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302232,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069608,"statfs":{"total":96636764160,"available":96327278592,"internally_reserved":0,"allocated":733184,"data_stored":629138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T15:42:32.591 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T15:42:32.684 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.682+0000 7fe4a9260640 1 Processor -- start 2024-09-13T15:42:32.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.682+0000 7fe4a9260640 1 -- start start 2024-09-13T15:42:32.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a9260640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a4096b40 0x7fe4a408f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a9260640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4a408f820 0x7fe4a408fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a9260640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe4a412c2b0 0x7fe4a412e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a9260640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a4090850 con 0x7fe4a408f820 2024-09-13T15:42:32.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a9260640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a4090170 con 0x7fe4a4096b40 2024-09-13T15:42:32.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a9260640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a40912e0 con 0x7fe4a412c2b0 2024-09-13T15:42:32.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a2d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a4096b40 0x7fe4a408f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4a408f820 0x7fe4a408fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a2d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a4096b40 0x7fe4a408f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:44400/0 (socket says 172.21.15.70:44400) 2024-09-13T15:42:32.685 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a2d9d640 1 -- 172.21.15.70:0/2900033028 learned_addr learned my addr 172.21.15.70:0/2900033028 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:32.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a359e640 1 --2- 172.21.15.70:0/2900033028 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe4a412c2b0 0x7fe4a412e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a259c640 1 -- 172.21.15.70:0/2900033028 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe4a412c2b0 msgr2=0x7fe4a412e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:32.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a259c640 1 --2- 172.21.15.70:0/2900033028 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe4a412c2b0 0x7fe4a412e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a259c640 1 -- 172.21.15.70:0/2900033028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a4096b40 msgr2=0x7fe4a408f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:32.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a259c640 1 --2- 172.21.15.70:0/2900033028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a4096b40 0x7fe4a408f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a259c640 1 -- 172.21.15.70:0/2900033028 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a4097730 con 0x7fe4a408f820 2024-09-13T15:42:32.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a2d9d640 1 --2- 172.21.15.70:0/2900033028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a4096b40 0x7fe4a408f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.686 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.683+0000 7fe4a359e640 1 --2- 172.21.15.70:0/2900033028 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe4a412c2b0 0x7fe4a412e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.684+0000 7fe4a259c640 1 --2- 172.21.15.70:0/2900033028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a408f820 0x7fe4a408fc30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009880 tx=0x7fe48c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174a07ad1d54188c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:32.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.684+0000 7fe4a1d9b640 1 -- 172.21.15.70:0/2900033028 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c02ac40 con 0x7fe4a408f820 2024-09-13T15:42:32.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.684+0000 7fe4a1d9b640 1 -- 172.21.15.70:0/2900033028 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c02ada0 con 0x7fe4a408f820 2024-09-13T15:42:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.684+0000 7fe4a1d9b640 1 -- 172.21.15.70:0/2900033028 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c032670 con 0x7fe4a408f820 2024-09-13T15:42:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.684+0000 7fe4a9260640 1 -- 172.21.15.70:0/2900033028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a408f820 msgr2=0x7fe4a408fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.684+0000 7fe4a9260640 1 --2- 172.21.15.70:0/2900033028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a408f820 0x7fe4a408fc30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009880 tx=0x7fe48c0090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.684+0000 7fe4a9260640 1 -- 172.21.15.70:0/2900033028 shutdown_connections 2024-09-13T15:42:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.684+0000 7fe4a9260640 1 --2- 172.21.15.70:0/2900033028 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe4a412c2b0 0x7fe4a412e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.684+0000 7fe4a9260640 1 --2- 172.21.15.70:0/2900033028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a408f820 0x7fe4a408fc30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.684+0000 7fe4a9260640 1 --2- 172.21.15.70:0/2900033028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a4096b40 0x7fe4a408f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.684+0000 7fe4a9260640 1 -- 172.21.15.70:0/2900033028 >> 172.21.15.70:0/2900033028 conn(0x7fe4a4097c30 msgr2=0x7fe4a4098030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.685+0000 7fe4a9260640 1 -- 172.21.15.70:0/2900033028 shutdown_connections 2024-09-13T15:42:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.685+0000 7fe4a9260640 1 -- 172.21.15.70:0/2900033028 wait complete. 2024-09-13T15:42:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.685+0000 7fe4a9260640 1 Processor -- start 2024-09-13T15:42:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.685+0000 7fe4a9260640 1 -- start start 2024-09-13T15:42:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a9260640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a408f820 0x7fe4a408e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a9260640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe4a4096b40 0x7fe4a406dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a9260640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a412c2b0 0x7fe4a406e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a9260640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a40899b0 con 0x7fe4a412c2b0 2024-09-13T15:42:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a9260640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a40897c0 con 0x7fe4a408f820 2024-09-13T15:42:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a9260640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a40883a0 con 0x7fe4a4096b40 2024-09-13T15:42:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a259c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe4a4096b40 0x7fe4a406dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a2d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a408f820 0x7fe4a408e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a412c2b0 0x7fe4a406e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a259c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe4a4096b40 0x7fe4a406dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41720/0 (socket says 172.21.15.70:41720) 2024-09-13T15:42:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a412c2b0 0x7fe4a406e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43978/0 (socket says 172.21.15.70:43978) 2024-09-13T15:42:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a2d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a408f820 0x7fe4a408e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:44414/0 (socket says 172.21.15.70:44414) 2024-09-13T15:42:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a259c640 1 -- 172.21.15.70:0/352397737 learned_addr learned my addr 172.21.15.70:0/352397737 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a259c640 1 -- 172.21.15.70:0/352397737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a408f820 msgr2=0x7fe4a408e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a259c640 1 --2- 172.21.15.70:0/352397737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a408f820 0x7fe4a408e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a259c640 1 -- 172.21.15.70:0/352397737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a412c2b0 msgr2=0x7fe4a406e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a259c640 1 --2- 172.21.15.70:0/352397737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a412c2b0 0x7fe4a406e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a259c640 1 -- 172.21.15.70:0/352397737 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe494009450 con 0x7fe4a4096b40 2024-09-13T15:42:32.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a2d9d640 1 --2- 172.21.15.70:0/352397737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a408f820 0x7fe4a408e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a359e640 1 --2- 172.21.15.70:0/352397737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a412c2b0 0x7fe4a406e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.686+0000 7fe4a259c640 1 --2- 172.21.15.70:0/352397737 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe4a4096b40 0x7fe4a406dc40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009360 tx=0x7fe48c032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:32.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.687+0000 7fe49affd640 1 -- 172.21.15.70:0/352397737 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c03d720 con 0x7fe4a4096b40 2024-09-13T15:42:32.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.687+0000 7fe49affd640 1 -- 172.21.15.70:0/352397737 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c03d880 con 0x7fe4a4096b40 2024-09-13T15:42:32.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.687+0000 7fe4a9260640 1 -- 172.21.15.70:0/352397737 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49c009a20 con 0x7fe4a4096b40 2024-09-13T15:42:32.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.687+0000 7fe49affd640 1 -- 172.21.15.70:0/352397737 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c03dc40 con 0x7fe4a4096b40 2024-09-13T15:42:32.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.687+0000 7fe4a9260640 1 -- 172.21.15.70:0/352397737 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48c009390 con 0x7fe4a4096b40 2024-09-13T15:42:32.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.688+0000 7fe49affd640 1 -- 172.21.15.70:0/352397737 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fe48c039040 con 0x7fe4a4096b40 2024-09-13T15:42:32.692 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.688+0000 7fe4a9260640 1 -- 172.21.15.70:0/352397737 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a4088f40 con 0x7fe4a4096b40 2024-09-13T15:42:32.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.688+0000 7fe49affd640 1 -- 172.21.15.70:0/352397737 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7fe48c082e40 con 0x7fe4a4096b40 2024-09-13T15:42:32.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.691+0000 7fe49affd640 1 -- 172.21.15.70:0/352397737 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fe48c05df80 con 0x7fe4a4096b40 2024-09-13T15:42:32.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.818+0000 7fe4a9260640 1 -- 172.21.15.70:0/352397737 --> v1:172.21.15.70:6816/32242 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4a4089140 con 0x7fe48404cb70 2024-09-13T15:42:32.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.819+0000 7fe49affd640 1 -- 172.21.15.70:0/352397737 <== mgr.4104 v1:172.21.15.70:6816/32242 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34969 (unknown 2965378022 0 3392297555) 0x7fe4a4089140 con 0x7fe48404cb70 2024-09-13T15:42:32.821 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:42:32.821 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-13T15:42:32.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.821+0000 7fe4a9260640 1 -- 172.21.15.70:0/352397737 >> v1:172.21.15.70:6816/32242 conn(0x7fe48404cb70 legacy=0x7fe48404f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.821+0000 7fe4a9260640 1 -- 172.21.15.70:0/352397737 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe4a4096b40 msgr2=0x7fe4a406dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.821+0000 7fe4a9260640 1 --2- 172.21.15.70:0/352397737 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe4a4096b40 0x7fe4a406dc40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009360 tx=0x7fe48c032000 comp rx=0 tx=0).stop 2024-09-13T15:42:32.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.821+0000 7fe4a9260640 1 -- 172.21.15.70:0/352397737 shutdown_connections 2024-09-13T15:42:32.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.821+0000 7fe4a9260640 1 --2- 172.21.15.70:0/352397737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4a412c2b0 0x7fe4a406e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.822 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.821+0000 7fe4a9260640 1 --2- 172.21.15.70:0/352397737 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe4a4096b40 0x7fe4a406dc40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.821+0000 7fe4a9260640 1 --2- 172.21.15.70:0/352397737 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe4a408f820 0x7fe4a408e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.821+0000 7fe4a9260640 1 -- 172.21.15.70:0/352397737 >> 172.21.15.70:0/352397737 conn(0x7fe4a4097c30 msgr2=0x7fe4a412b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:32.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.821+0000 7fe4a9260640 1 -- 172.21.15.70:0/352397737 shutdown_connections 2024-09-13T15:42:32.823 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.821+0000 7fe4a9260640 1 -- 172.21.15.70:0/352397737 wait complete. 2024-09-13T15:42:32.832 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-13T15:42:30.971937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1207784,"kb_used_data":1712,"kb_used_omap":6,"kb_used_meta":1206009,"kb_avail":376279576,"statfs":{"total":386547056640,"available":385310285824,"internally_reserved":0,"allocated":1753088,"data_stored":1335854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1234953568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.262342"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.720294+0000","last_change":"2024-09-13T15:42:29.720527+0000","last_active":"2024-09-13T15:42:29.720294+0000","last_peered":"2024-09-13T15:42:29.720294+0000","last_clean":"2024-09-13T15:42:29.720294+0000","last_became_active":"2024-09-13T15:42:27.716998+0000","last_became_peered":"2024-09-13T15:42:27.716998+0000","last_unstale":"2024-09-13T15:42:29.720294+0000","last_undegraded":"2024-09-13T15:42:29.720294+0000","last_fullsized":"2024-09-13T15:42:29.720294+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:22.475936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035985100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.717506+0000","last_change":"2024-09-13T15:42:29.717707+0000","last_active":"2024-09-13T15:42:29.717506+0000","last_peered":"2024-09-13T15:42:29.717506+0000","last_clean":"2024-09-13T15:42:29.717506+0000","last_became_active":"2024-09-13T15:42:27.718860+0000","last_became_peered":"2024-09-13T15:42:27.718860+0000","last_unstale":"2024-09-13T15:42:29.717506+0000","last_undegraded":"2024-09-13T15:42:29.717506+0000","last_fullsized":"2024-09-13T15:42:29.717506+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:19.555928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054553999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.721176+0000","last_change":"2024-09-13T15:42:29.721358+0000","last_active":"2024-09-13T15:42:29.721176+0000","last_peered":"2024-09-13T15:42:29.721176+0000","last_clean":"2024-09-13T15:42:29.721176+0000","last_became_active":"2024-09-13T15:42:27.717168+0000","last_became_peered":"2024-09-13T15:42:27.717168+0000","last_unstale":"2024-09-13T15:42:29.721176+0000","last_undegraded":"2024-09-13T15:42:29.721176+0000","last_fullsized":"2024-09-13T15:42:29.721176+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:15.334763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011766770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.717555+0000","last_change":"2024-09-13T15:42:29.717708+0000","last_active":"2024-09-13T15:42:29.717555+0000","last_peered":"2024-09-13T15:42:29.717555+0000","last_clean":"2024-09-13T15:42:29.717555+0000","last_became_active":"2024-09-13T15:42:27.719566+0000","last_became_peered":"2024-09-13T15:42:27.719566+0000","last_unstale":"2024-09-13T15:42:29.717555+0000","last_undegraded":"2024-09-13T15:42:29.717555+0000","last_fullsized":"2024-09-13T15:42:29.717555+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:59.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024452499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"16'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.721120+0000","last_change":"2024-09-13T15:42:29.721120+0000","last_active":"2024-09-13T15:42:29.721120+0000","last_peered":"2024-09-13T15:42:29.721120+0000","last_clean":"2024-09-13T15:42:29.721120+0000","last_became_active":"2024-09-13T15:42:27.718029+0000","last_became_peered":"2024-09-13T15:42:27.718029+0000","last_unstale":"2024-09-13T15:42:29.721120+0000","last_undegraded":"2024-09-13T15:42:29.721120+0000","last_fullsized":"2024-09-13T15:42:29.721120+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:44.778809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094542300000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.717199+0000","last_change":"2024-09-13T15:42:29.717606+0000","last_active":"2024-09-13T15:42:29.717199+0000","last_peered":"2024-09-13T15:42:29.717199+0000","last_clean":"2024-09-13T15:42:29.717199+0000","last_became_active":"2024-09-13T15:42:27.715108+0000","last_became_peered":"2024-09-13T15:42:27.715108+0000","last_unstale":"2024-09-13T15:42:29.717199+0000","last_undegraded":"2024-09-13T15:42:29.717199+0000","last_fullsized":"2024-09-13T15:42:29.717199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:34.509753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089033199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.720264+0000","last_change":"2024-09-13T15:42:29.720489+0000","last_active":"2024-09-13T15:42:29.720264+0000","last_peered":"2024-09-13T15:42:29.720264+0000","last_clean":"2024-09-13T15:42:29.720264+0000","last_became_active":"2024-09-13T15:42:27.718347+0000","last_became_peered":"2024-09-13T15:42:27.718347+0000","last_unstale":"2024-09-13T15:42:29.720264+0000","last_undegraded":"2024-09-13T15:42:29.720264+0000","last_fullsized":"2024-09-13T15:42:29.720264+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:16.064731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034895899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"14'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-13T15:42:29.720963+0000","last_change":"2024-09-13T15:42:29.721205+0000","last_active":"2024-09-13T15:42:29.720963+0000","last_peered":"2024-09-13T15:42:29.720963+0000","last_clean":"2024-09-13T15:42:29.720963+0000","last_became_active":"2024-09-13T15:42:27.720033+0000","last_became_peered":"2024-09-13T15:42:27.720033+0000","last_unstale":"2024-09-13T15:42:29.720963+0000","last_undegraded":"2024-09-13T15:42:29.720963+0000","last_fullsized":"2024-09-13T15:42:29.720963+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:36.892284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276560000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"11'52","reported_seq":80,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T15:42:28.713720+0000","last_change":"2024-09-13T15:42:22.700543+0000","last_active":"2024-09-13T15:42:28.713720+0000","last_peered":"2024-09-13T15:42:28.713720+0000","last_clean":"2024-09-13T15:42:28.713720+0000","last_became_active":"2024-09-13T15:42:22.700226+0000","last_became_peered":"2024-09-13T15:42:22.700226+0000","last_unstale":"2024-09-13T15:42:28.713720+0000","last_undegraded":"2024-09-13T15:42:28.713720+0000","last_fullsized":"2024-09-13T15:42:28.713720+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_clean_scrub_stamp":"2024-09-13T15:42:21.689345+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:02.263603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301660,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070180,"statfs":{"total":96636764160,"available":96327864320,"internally_reserved":0,"allocated":143360,"data_stored":38789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]}]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301656,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070184,"statfs":{"total":96636764160,"available":96327868416,"internally_reserved":0,"allocated":139264,"data_stored":38770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27700000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302236,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069604,"statfs":{"total":96636764160,"available":96327274496,"internally_reserved":0,"allocated":737280,"data_stored":629157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302232,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069608,"statfs":{"total":96636764160,"available":96327278592,"internally_reserved":0,"allocated":733184,"data_stored":629138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T15:42:32.833 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T15:42:32.833 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T15:42:32.833 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T15:42:32.834 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T15:42:32.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.923+0000 7f04dfc0f640 1 Processor -- start 2024-09-13T15:42:32.925 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.923+0000 7f04dfc0f640 1 -- start start 2024-09-13T15:42:32.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dfc0f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f04d8096b40 0x7f04d808f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dfc0f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04d808f820 0x7f04d808fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dfc0f640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f04d812c2b0 0x7f04d812e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dfc0f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d8090850 con 0x7f04d8096b40 2024-09-13T15:42:32.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dfc0f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d8090170 con 0x7f04d808f820 2024-09-13T15:42:32.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dfc0f640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04d80912e0 con 0x7f04d812c2b0 2024-09-13T15:42:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dd1aa640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04d808f820 0x7f04d808fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dd9ab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f04d8096b40 0x7f04d808f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04de1ac640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f04d812c2b0 0x7f04d812e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dd1aa640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04d808f820 0x7f04d808fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:44424/0 (socket says 172.21.15.70:44424) 2024-09-13T15:42:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dd1aa640 1 -- 172.21.15.70:0/3077795301 learned_addr learned my addr 172.21.15.70:0/3077795301 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dd1aa640 1 -- 172.21.15.70:0/3077795301 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f04d812c2b0 msgr2=0x7f04d812e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dd1aa640 1 --2- 172.21.15.70:0/3077795301 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f04d812c2b0 0x7f04d812e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dd1aa640 1 -- 172.21.15.70:0/3077795301 >> v2:172.21.15.70:3300/0 conn(0x7f04d8096b40 msgr2=0x7f04d808f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dd1aa640 1 --2- 172.21.15.70:0/3077795301 >> v2:172.21.15.70:3300/0 conn(0x7f04d8096b40 0x7f04d808f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dd1aa640 1 -- 172.21.15.70:0/3077795301 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d8097730 con 0x7f04d808f820 2024-09-13T15:42:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04dd9ab640 1 --2- 172.21.15.70:0/3077795301 >> v2:172.21.15.70:3300/0 conn(0x7f04d8096b40 0x7f04d808f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.924+0000 7f04de1ac640 1 --2- 172.21.15.70:0/3077795301 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f04d812c2b0 0x7f04d812e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.925+0000 7f04dd1aa640 1 --2- 172.21.15.70:0/3077795301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04d808f820 0x7f04d808fc30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009910 tx=0x7f04c80091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb22cba0e6450caf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.925+0000 7f04dc9a9640 1 -- 172.21.15.70:0/3077795301 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c802ad20 con 0x7f04d808f820 2024-09-13T15:42:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.925+0000 7f04dc9a9640 1 -- 172.21.15.70:0/3077795301 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c802ae80 con 0x7f04d808f820 2024-09-13T15:42:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.925+0000 7f04dc9a9640 1 -- 172.21.15.70:0/3077795301 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8032670 con 0x7f04d808f820 2024-09-13T15:42:32.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.926+0000 7f04dfc0f640 1 -- 172.21.15.70:0/3077795301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04d808f820 msgr2=0x7f04d808fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.926+0000 7f04dfc0f640 1 --2- 172.21.15.70:0/3077795301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04d808f820 0x7f04d808fc30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009910 tx=0x7f04c80091d0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.926+0000 7f04dfc0f640 1 -- 172.21.15.70:0/3077795301 shutdown_connections 2024-09-13T15:42:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.926+0000 7f04dfc0f640 1 --2- 172.21.15.70:0/3077795301 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f04d812c2b0 0x7f04d812e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.926+0000 7f04dfc0f640 1 --2- 172.21.15.70:0/3077795301 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04d808f820 0x7f04d808fc30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.926+0000 7f04dfc0f640 1 --2- 172.21.15.70:0/3077795301 >> v2:172.21.15.70:3300/0 conn(0x7f04d8096b40 0x7f04d808f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.926+0000 7f04dfc0f640 1 -- 172.21.15.70:0/3077795301 >> 172.21.15.70:0/3077795301 conn(0x7f04d8097c30 msgr2=0x7f04d8098030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.926+0000 7f04dfc0f640 1 -- 172.21.15.70:0/3077795301 shutdown_connections 2024-09-13T15:42:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.926+0000 7f04dfc0f640 1 -- 172.21.15.70:0/3077795301 wait complete. 2024-09-13T15:42:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.926+0000 7f04dfc0f640 1 Processor -- start 2024-09-13T15:42:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dfc0f640 1 -- start start 2024-09-13T15:42:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dfc0f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04d808f820 0x7f04d806df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dfc0f640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f04d8096b40 0x7f04d806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dfc0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04d812c2b0 0x7f04d806e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dfc0f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d8089770 con 0x7f04d812c2b0 2024-09-13T15:42:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dfc0f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d8087f60 con 0x7f04d808f820 2024-09-13T15:42:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dfc0f640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04d8088760 con 0x7f04d8096b40 2024-09-13T15:42:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dd9ab640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04d808f820 0x7f04d806df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dd1aa640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f04d8096b40 0x7f04d806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dd9ab640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04d808f820 0x7f04d806df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:44438/0 (socket says 172.21.15.70:44438) 2024-09-13T15:42:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dd9ab640 1 -- 172.21.15.70:0/1612343034 learned_addr learned my addr 172.21.15.70:0/1612343034 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04de1ac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04d812c2b0 0x7f04d806e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dd1aa640 1 -- 172.21.15.70:0/1612343034 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04d808f820 msgr2=0x7f04d806df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dd1aa640 1 --2- 172.21.15.70:0/1612343034 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04d808f820 0x7f04d806df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dd1aa640 1 -- 172.21.15.70:0/1612343034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04d812c2b0 msgr2=0x7f04d806e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dd1aa640 1 --2- 172.21.15.70:0/1612343034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04d812c2b0 0x7f04d806e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dd1aa640 1 -- 172.21.15.70:0/1612343034 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d4009390 con 0x7f04d8096b40 2024-09-13T15:42:32.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dd9ab640 1 --2- 172.21.15.70:0/1612343034 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04d808f820 0x7f04d806df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04de1ac640 1 --2- 172.21.15.70:0/1612343034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04d812c2b0 0x7f04d806e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:32.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.927+0000 7f04dd1aa640 1 --2- 172.21.15.70:0/1612343034 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f04d8096b40 0x7f04d806e440 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f04d8064ad0 tx=0x7f04c8032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:32.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.928+0000 7f04cdffb640 1 -- 172.21.15.70:0/1612343034 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c803b610 con 0x7f04d8096b40 2024-09-13T15:42:32.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.928+0000 7f04cdffb640 1 -- 172.21.15.70:0/1612343034 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c803b790 con 0x7f04d8096b40 2024-09-13T15:42:32.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.928+0000 7f04cdffb640 1 -- 172.21.15.70:0/1612343034 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c803bb50 con 0x7f04d8096b40 2024-09-13T15:42:32.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.928+0000 7f04dfc0f640 1 -- 172.21.15.70:0/1612343034 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d0009a20 con 0x7f04d8096b40 2024-09-13T15:42:32.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.928+0000 7f04dfc0f640 1 -- 172.21.15.70:0/1612343034 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c8009450 con 0x7f04d8096b40 2024-09-13T15:42:32.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.929+0000 7f04cdffb640 1 -- 172.21.15.70:0/1612343034 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f04c8039040 con 0x7f04d8096b40 2024-09-13T15:42:32.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.929+0000 7f04dfc0f640 1 -- 172.21.15.70:0/1612343034 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d805ed70 con 0x7f04d8096b40 2024-09-13T15:42:32.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.929+0000 7f04cdffb640 1 -- 172.21.15.70:0/1612343034 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f04c8082190 con 0x7f04d8096b40 2024-09-13T15:42:32.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:32.932+0000 7f04cdffb640 1 -- 172.21.15.70:0/1612343034 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f04c805d2d0 con 0x7f04d8096b40 2024-09-13T15:42:33.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.084+0000 7f04dfc0f640 1 -- 172.21.15.70:0/1612343034 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f04d8093cf0 con 0x7f04d8096b40 2024-09-13T15:42:33.086 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.085+0000 7f04cdffb640 1 -- 172.21.15.70:0/1612343034 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f04c803f040 con 0x7f04d8096b40 2024-09-13T15:42:33.086 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:42:33.086 INFO:teuthology.orchestra.run.smithi070.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T15:42:33.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.086+0000 7f04dfc0f640 1 -- 172.21.15.70:0/1612343034 >> v1:172.21.15.70:6816/32242 conn(0x7f04bc04cb70 legacy=0x7f04bc04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:33.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.086+0000 7f04dfc0f640 1 -- 172.21.15.70:0/1612343034 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f04d8096b40 msgr2=0x7f04d806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:33.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.086+0000 7f04dfc0f640 1 --2- 172.21.15.70:0/1612343034 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f04d8096b40 0x7f04d806e440 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f04d8064ad0 tx=0x7f04c8032000 comp rx=0 tx=0).stop 2024-09-13T15:42:33.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.086+0000 7f04dfc0f640 1 -- 172.21.15.70:0/1612343034 shutdown_connections 2024-09-13T15:42:33.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.086+0000 7f04dfc0f640 1 --2- 172.21.15.70:0/1612343034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04d812c2b0 0x7f04d806e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:33.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.086+0000 7f04dfc0f640 1 --2- 172.21.15.70:0/1612343034 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f04d8096b40 0x7f04d806e440 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:33.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.086+0000 7f04dfc0f640 1 --2- 172.21.15.70:0/1612343034 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f04d808f820 0x7f04d806df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:33.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.086+0000 7f04dfc0f640 1 -- 172.21.15.70:0/1612343034 >> 172.21.15.70:0/1612343034 conn(0x7f04d8097c30 msgr2=0x7f04d812b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:33.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.087+0000 7f04dfc0f640 1 -- 172.21.15.70:0/1612343034 shutdown_connections 2024-09-13T15:42:33.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.087+0000 7f04dfc0f640 1 -- 172.21.15.70:0/1612343034 wait complete. 2024-09-13T15:42:33.098 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T15:42:33.098 INFO:teuthology.run_tasks:Running task exec... 2024-09-13T15:42:33.107 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T15:42:33.107 INFO:teuthology.task.exec:Running commands on role osd.0 host ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T15:42:33.108 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd set-require-min-compat-client quincy' 2024-09-13T15:42:33.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.197+0000 7f4c19f9d640 1 Processor -- start 2024-09-13T15:42:33.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.197+0000 7f4c19f9d640 1 -- start start 2024-09-13T15:42:33.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.197+0000 7f4c19f9d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4c140959f0 0x7f4c14095dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:33.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.197+0000 7f4c19f9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4c1412f8b0 0x7f4c14131c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:33.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.197+0000 7f4c19f9d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c14068390 con 0x7f4c14096300 2024-09-13T15:42:33.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.197+0000 7f4c19f9d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1406f040 con 0x7f4c1412f8b0 2024-09-13T15:42:33.199 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.197+0000 7f4c19f9d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c14094470 con 0x7f4c140959f0 2024-09-13T15:42:33.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.197+0000 7f4c137fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4c140959f0 0x7f4c14095dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:33.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.197+0000 7f4c13fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4c1412f8b0 0x7f4c14131c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:33.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.197+0000 7f4c12ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4c14096300 0x7f4c1412e190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38790/0 (socket says 172.21.15.70:38790) 2024-09-13T15:42:33.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.197+0000 7f4c137fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4c140959f0 0x7f4c14095dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41752/0 (socket says 172.21.15.70:41752) 2024-09-13T15:42:33.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.197+0000 7f4c12ffd640 1 -- 172.21.15.70:0/807150746 learned_addr learned my addr 172.21.15.70:0/807150746 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:42:33.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.198+0000 7f4c137fe640 1 -- 172.21.15.70:0/807150746 >> v2:172.21.15.70:3300/0 conn(0x7f4c1412f8b0 msgr2=0x7f4c14131c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:33.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.198+0000 7f4c137fe640 1 --2- 172.21.15.70:0/807150746 >> v2:172.21.15.70:3300/0 conn(0x7f4c1412f8b0 0x7f4c14131c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:33.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.198+0000 7f4c137fe640 1 -- 172.21.15.70:0/807150746 >> v1:172.21.15.70:6789/0 conn(0x7f4c14096300 legacy=0x7f4c1412e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:33.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.198+0000 7f4c127fc640 1 -- 172.21.15.70:0/807150746 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156811573 0 0) 0x7f4c14068390 con 0x7f4c14096300 2024-09-13T15:42:33.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.198+0000 7f4c137fe640 1 -- 172.21.15.70:0/807150746 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c14067d00 con 0x7f4c140959f0 2024-09-13T15:42:33.200 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.198+0000 7f4c13fff640 1 --2- 172.21.15.70:0/807150746 >> v2:172.21.15.70:3300/0 conn(0x7f4c1412f8b0 0x7f4c14131c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.198+0000 7f4c137fe640 1 --2- 172.21.15.70:0/807150746 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4c140959f0 0x7f4c14095dc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009850 tx=0x7f4c040090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37200759d305e9dc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.198+0000 7f4c127fc640 1 -- 172.21.15.70:0/807150746 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0402ad00 con 0x7f4c140959f0 2024-09-13T15:42:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.198+0000 7f4c127fc640 1 -- 172.21.15.70:0/807150746 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0402ae60 con 0x7f4c140959f0 2024-09-13T15:42:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.198+0000 7f4c127fc640 1 -- 172.21.15.70:0/807150746 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04032670 con 0x7f4c140959f0 2024-09-13T15:42:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.199+0000 7f4c19f9d640 1 -- 172.21.15.70:0/807150746 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4c140959f0 msgr2=0x7f4c14095dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.199+0000 7f4c19f9d640 1 --2- 172.21.15.70:0/807150746 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4c140959f0 0x7f4c14095dc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009850 tx=0x7f4c040090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.199+0000 7f4c19f9d640 1 -- 172.21.15.70:0/807150746 shutdown_connections 2024-09-13T15:42:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.199+0000 7f4c19f9d640 1 --2- 172.21.15.70:0/807150746 >> v2:172.21.15.70:3300/0 conn(0x7f4c1412f8b0 0x7f4c14131c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.199+0000 7f4c19f9d640 1 --2- 172.21.15.70:0/807150746 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4c140959f0 0x7f4c14095dc0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.199+0000 7f4c19f9d640 1 -- 172.21.15.70:0/807150746 >> 172.21.15.70:0/807150746 conn(0x7f4c14096ae0 msgr2=0x7f4c14096ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.199+0000 7f4c19f9d640 1 -- 172.21.15.70:0/807150746 shutdown_connections 2024-09-13T15:42:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.199+0000 7f4c19f9d640 1 -- 172.21.15.70:0/807150746 wait complete. 2024-09-13T15:42:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.200+0000 7f4c19f9d640 1 Processor -- start 2024-09-13T15:42:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.200+0000 7f4c19f9d640 1 -- start start 2024-09-13T15:42:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.200+0000 7f4c19f9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c140959f0 0x7f4c1412efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.200+0000 7f4c19f9d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4c14096300 0x7f4c1411f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.200+0000 7f4c19f9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c1412f8b0 0x7f4c1411f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.200+0000 7f4c19f9d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c14121f60 con 0x7f4c1412f8b0 2024-09-13T15:42:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.200+0000 7f4c19f9d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c14122330 con 0x7f4c140959f0 2024-09-13T15:42:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.200+0000 7f4c19f9d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c14120f80 con 0x7f4c14096300 2024-09-13T15:42:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.200+0000 7f4c12ffd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4c14096300 0x7f4c1411f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.200+0000 7f4c137fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c140959f0 0x7f4c1412efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.200+0000 7f4c13fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c1412f8b0 0x7f4c1411f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.200+0000 7f4c12ffd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4c14096300 0x7f4c1411f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:41758/0 (socket says 172.21.15.70:41758) 2024-09-13T15:42:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.200+0000 7f4c12ffd640 1 -- 172.21.15.70:0/385758692 learned_addr learned my addr 172.21.15.70:0/385758692 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.201+0000 7f4c12ffd640 1 -- 172.21.15.70:0/385758692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c140959f0 msgr2=0x7f4c1412efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.201+0000 7f4c12ffd640 1 --2- 172.21.15.70:0/385758692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c140959f0 0x7f4c1412efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.201+0000 7f4c12ffd640 1 -- 172.21.15.70:0/385758692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c1412f8b0 msgr2=0x7f4c1411f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.201+0000 7f4c12ffd640 1 --2- 172.21.15.70:0/385758692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c1412f8b0 0x7f4c1411f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.201+0000 7f4c12ffd640 1 -- 172.21.15.70:0/385758692 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c009a20 con 0x7f4c14096300 2024-09-13T15:42:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.201+0000 7f4c13fff640 1 --2- 172.21.15.70:0/385758692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c1412f8b0 0x7f4c1411f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.201+0000 7f4c12ffd640 1 --2- 172.21.15.70:0/385758692 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4c14096300 0x7f4c1411f0c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc001eb0 tx=0x7f4bfc007ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.201+0000 7f4c0b7fe640 1 -- 172.21.15.70:0/385758692 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc005aa0 con 0x7f4c14096300 2024-09-13T15:42:33.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.201+0000 7f4c0b7fe640 1 -- 172.21.15.70:0/385758692 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bfc005c20 con 0x7f4c14096300 2024-09-13T15:42:33.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.201+0000 7f4c19f9d640 1 -- 172.21.15.70:0/385758692 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c04009390 con 0x7f4c14096300 2024-09-13T15:42:33.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.201+0000 7f4c0b7fe640 1 -- 172.21.15.70:0/385758692 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc007320 con 0x7f4c14096300 2024-09-13T15:42:33.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.201+0000 7f4c19f9d640 1 -- 172.21.15.70:0/385758692 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1412f4e0 con 0x7f4c14096300 2024-09-13T15:42:33.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.202+0000 7f4c19f9d640 1 -- 172.21.15.70:0/385758692 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1412f700 con 0x7f4c14096300 2024-09-13T15:42:33.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.202+0000 7f4c0b7fe640 1 -- 172.21.15.70:0/385758692 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7f4bfc01b3b0 con 0x7f4c14096300 2024-09-13T15:42:33.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.202+0000 7f4c0b7fe640 1 -- 172.21.15.70:0/385758692 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2864+0+0 (secure 0 0 0) 0x7f4bfc062640 con 0x7f4c14096300 2024-09-13T15:42:33.206 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.206+0000 7f4c0b7fe640 1 -- 172.21.15.70:0/385758692 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7f4bfc03d7e0 con 0x7f4c14096300 2024-09-13T15:42:33.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.344+0000 7f4c19f9d640 1 -- 172.21.15.70:0/385758692 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "quincy"} v 0) v1 -- 0x7f4c1404bf60 con 0x7f4c14096300 2024-09-13T15:42:33.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.725+0000 7f4c0b7fe640 1 -- 172.21.15.70:0/385758692 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "quincy"}]=0 set require_min_compat_client to quincy v17) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4bfc017070 con 0x7f4c14096300 2024-09-13T15:42:33.727 INFO:teuthology.orchestra.run.smithi070.stderr:set require_min_compat_client to quincy 2024-09-13T15:42:33.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.727+0000 7f4c19f9d640 1 -- 172.21.15.70:0/385758692 >> v1:172.21.15.70:6816/32242 conn(0x7f4bf404cb70 legacy=0x7f4bf404f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:33.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.727+0000 7f4c19f9d640 1 -- 172.21.15.70:0/385758692 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4c14096300 msgr2=0x7f4c1411f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:33.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.727+0000 7f4c19f9d640 1 --2- 172.21.15.70:0/385758692 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4c14096300 0x7f4c1411f0c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc001eb0 tx=0x7f4bfc007ce0 comp rx=0 tx=0).stop 2024-09-13T15:42:33.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.727+0000 7f4c19f9d640 1 -- 172.21.15.70:0/385758692 shutdown_connections 2024-09-13T15:42:33.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.727+0000 7f4c19f9d640 1 --2- 172.21.15.70:0/385758692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c1412f8b0 0x7f4c1411f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:33.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.727+0000 7f4c19f9d640 1 --2- 172.21.15.70:0/385758692 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4c14096300 0x7f4c1411f0c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:33.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.727+0000 7f4c19f9d640 1 --2- 172.21.15.70:0/385758692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4c140959f0 0x7f4c1412efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:33.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.727+0000 7f4c19f9d640 1 -- 172.21.15.70:0/385758692 >> 172.21.15.70:0/385758692 conn(0x7f4c14096ae0 msgr2=0x7f4c14089fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:33.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.727+0000 7f4c19f9d640 1 -- 172.21.15.70:0/385758692 shutdown_connections 2024-09-13T15:42:33.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:33.727+0000 7f4c19f9d640 1 -- 172.21.15.70:0/385758692 wait complete. 2024-09-13T15:42:33.739 INFO:teuthology.run_tasks:Running task print... 2024-09-13T15:42:34.082 INFO:teuthology.task.print:**** done ceph 2024-09-13T15:42:34.083 INFO:teuthology.run_tasks:Running task print... 2024-09-13T15:42:34.489 INFO:teuthology.task.print:*** upgrading, no cephfs present 2024-09-13T15:42:34.490 INFO:teuthology.run_tasks:Running task exec... 2024-09-13T15:42:34.807 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T15:42:34.807 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T15:42:34.807 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs dump' 2024-09-13T15:42:34.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.906+0000 7fce20b1d640 1 Processor -- start 2024-09-13T15:42:34.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.906+0000 7fce20b1d640 1 -- start start 2024-09-13T15:42:34.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.907+0000 7fce20b1d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fce1c068870 0x7fce1c12f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:34.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.907+0000 7fce20b1d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce1c068cf0 0x7fce1c131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:34.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.907+0000 7fce20b1d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce1c069e20 con 0x7fce1c098d30 2024-09-13T15:42:34.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.907+0000 7fce20b1d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c0691e0 con 0x7fce1c068cf0 2024-09-13T15:42:34.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.907+0000 7fce20b1d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c09ac10 con 0x7fce1c068870 2024-09-13T15:42:34.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.907+0000 7fce1a59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fce1c098d30 0x7fce1c068160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40890/0 (socket says 172.21.15.70:40890) 2024-09-13T15:42:34.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.907+0000 7fce1a59c640 1 -- 172.21.15.70:0/389208271 learned_addr learned my addr 172.21.15.70:0/389208271 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:42:34.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.907+0000 7fce19d9b640 1 --2- 172.21.15.70:0/389208271 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fce1c068870 0x7fce1c12f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:34.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.907+0000 7fce1ad9d640 1 --2- 172.21.15.70:0/389208271 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce1c068cf0 0x7fce1c131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:34.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.907+0000 7fce1959a640 1 -- 172.21.15.70:0/389208271 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985818515 0 0) 0x7fce1c069e20 con 0x7fce1c098d30 2024-09-13T15:42:34.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.907+0000 7fce1959a640 1 -- 172.21.15.70:0/389208271 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce080034e0 con 0x7fce1c098d30 2024-09-13T15:42:34.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.907+0000 7fce19d9b640 1 -- 172.21.15.70:0/389208271 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce1c068cf0 msgr2=0x7fce1c131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:42:34.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.907+0000 7fce19d9b640 1 --2- 172.21.15.70:0/389208271 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce1c068cf0 0x7fce1c131a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:34.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.907+0000 7fce19d9b640 1 -- 172.21.15.70:0/389208271 >> v1:172.21.15.70:6789/0 conn(0x7fce1c098d30 legacy=0x7fce1c068160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:34.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.908+0000 7fce19d9b640 1 -- 172.21.15.70:0/389208271 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c132f50 con 0x7fce1c068870 2024-09-13T15:42:34.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.908+0000 7fce19d9b640 1 --2- 172.21.15.70:0/389208271 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fce1c068870 0x7fce1c12f4b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fce04009880 tx=0x7fce040090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64ca008b80ad3a74 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:34.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.908+0000 7fce1959a640 1 -- 172.21.15.70:0/389208271 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0402acd0 con 0x7fce1c068870 2024-09-13T15:42:34.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.908+0000 7fce1959a640 1 -- 172.21.15.70:0/389208271 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0402ae30 con 0x7fce1c068870 2024-09-13T15:42:34.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.908+0000 7fce1959a640 1 -- 172.21.15.70:0/389208271 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce04032650 con 0x7fce1c068870 2024-09-13T15:42:34.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.909+0000 7fce20b1d640 1 -- 172.21.15.70:0/389208271 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fce1c068870 msgr2=0x7fce1c12f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:34.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.909+0000 7fce20b1d640 1 --2- 172.21.15.70:0/389208271 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fce1c068870 0x7fce1c12f4b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fce04009880 tx=0x7fce040090f0 comp rx=0 tx=0).stop 2024-09-13T15:42:34.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.909+0000 7fce20b1d640 1 -- 172.21.15.70:0/389208271 shutdown_connections 2024-09-13T15:42:34.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.909+0000 7fce20b1d640 1 --2- 172.21.15.70:0/389208271 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce1c068cf0 0x7fce1c131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:34.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.909+0000 7fce20b1d640 1 --2- 172.21.15.70:0/389208271 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fce1c068870 0x7fce1c12f4b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:34.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.909+0000 7fce20b1d640 1 -- 172.21.15.70:0/389208271 >> 172.21.15.70:0/389208271 conn(0x7fce1c09df70 msgr2=0x7fce1c099e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:34.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.909+0000 7fce20b1d640 1 -- 172.21.15.70:0/389208271 shutdown_connections 2024-09-13T15:42:34.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.909+0000 7fce20b1d640 1 -- 172.21.15.70:0/389208271 wait complete. 2024-09-13T15:42:34.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.910+0000 7fce20b1d640 1 Processor -- start 2024-09-13T15:42:34.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.910+0000 7fce20b1d640 1 -- start start 2024-09-13T15:42:34.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.910+0000 7fce20b1d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fce1c068870 0x7fce1c131720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:34.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.910+0000 7fce20b1d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce1c068cf0 0x7fce1c08cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:34.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.910+0000 7fce20b1d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce1c098d30 0x7fce1c08d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:42:34.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.910+0000 7fce20b1d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c08f0a0 con 0x7fce1c068cf0 2024-09-13T15:42:34.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.910+0000 7fce20b1d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c0962c0 con 0x7fce1c098d30 2024-09-13T15:42:34.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.910+0000 7fce20b1d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c08fe50 con 0x7fce1c068870 2024-09-13T15:42:34.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.910+0000 7fce1a59c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fce1c068870 0x7fce1c131720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:34.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.910+0000 7fce19d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce1c068cf0 0x7fce1c08cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:34.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.910+0000 7fce19d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce1c068cf0 0x7fce1c08cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40314/0 (socket says 172.21.15.70:40314) 2024-09-13T15:42:34.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.910+0000 7fce1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce1c098d30 0x7fce1c08d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:42:34.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.910+0000 7fce1a59c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fce1c068870 0x7fce1c131720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:34076/0 (socket says 172.21.15.70:34076) 2024-09-13T15:42:34.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.910+0000 7fce19d9b640 1 -- 172.21.15.70:0/2597336187 learned_addr learned my addr 172.21.15.70:0/2597336187 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:42:34.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.911+0000 7fce1a59c640 1 -- 172.21.15.70:0/2597336187 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce1c098d30 msgr2=0x7fce1c08d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.911+0000 7fce1a59c640 1 --2- 172.21.15.70:0/2597336187 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce1c098d30 0x7fce1c08d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.911+0000 7fce1a59c640 1 -- 172.21.15.70:0/2597336187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce1c068cf0 msgr2=0x7fce1c08cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.911+0000 7fce1a59c640 1 --2- 172.21.15.70:0/2597336187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce1c068cf0 0x7fce1c08cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.911+0000 7fce1a59c640 1 -- 172.21.15.70:0/2597336187 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce04009390 con 0x7fce1c068870 2024-09-13T15:42:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.911+0000 7fce19d9b640 1 --2- 172.21.15.70:0/2597336187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce1c068cf0 0x7fce1c08cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:42:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.911+0000 7fce1a59c640 1 --2- 172.21.15.70:0/2597336187 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fce1c068870 0x7fce1c131720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fce1400cc20 tx=0x7fce14001a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:42:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.911+0000 7fce127fc640 1 -- 172.21.15.70:0/2597336187 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1400ce40 con 0x7fce1c068870 2024-09-13T15:42:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.911+0000 7fce127fc640 1 -- 172.21.15.70:0/2597336187 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1400a850 con 0x7fce1c068870 2024-09-13T15:42:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.911+0000 7fce20b1d640 1 -- 172.21.15.70:0/2597336187 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c090360 con 0x7fce1c068870 2024-09-13T15:42:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.911+0000 7fce127fc640 1 -- 172.21.15.70:0/2597336187 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1400ac50 con 0x7fce1c068870 2024-09-13T15:42:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.911+0000 7fce20b1d640 1 -- 172.21.15.70:0/2597336187 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1c08d5f0 con 0x7fce1c068870 2024-09-13T15:42:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.912+0000 7fce127fc640 1 -- 172.21.15.70:0/2597336187 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 5) v1 ==== 58551+0+0 (secure 0 0 0) 0x7fce14007300 con 0x7fce1c068870 2024-09-13T15:42:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.912+0000 7fce20b1d640 1 -- 172.21.15.70:0/2597336187 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1c04bf60 con 0x7fce1c068870 2024-09-13T15:42:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.912+0000 7fce127fc640 1 -- 172.21.15.70:0/2597336187 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2836+0+0 (secure 0 0 0) 0x7fce14063320 con 0x7fce1c068870 2024-09-13T15:42:34.919 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:34.916+0000 7fce127fc640 1 -- 172.21.15.70:0/2597336187 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+139858 (secure 0 0 0) 0x7fce1403e4f0 con 0x7fce1c068870 2024-09-13T15:42:35.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:35.067+0000 7fce20b1d640 1 -- 172.21.15.70:0/2597336187 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fce1c096940 con 0x7fce1c068870 2024-09-13T15:42:35.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:35.067+0000 7fce127fc640 1 -- 172.21.15.70:0/2597336187 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 1 v1) v1 ==== 75+0+358 (secure 0 0 0) 0x7fce14041d00 con 0x7fce1c068870 2024-09-13T15:42:35.069 INFO:teuthology.orchestra.run.smithi070.stdout:e1 2024-09-13T15:42:35.069 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T15:42:35.069 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-13T15:42:35.069 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: -1 2024-09-13T15:42:35.069 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:42:35.069 INFO:teuthology.orchestra.run.smithi070.stdout:No filesystems configured 2024-09-13T15:42:35.070 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 1 2024-09-13T15:42:35.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:35.069+0000 7fce20b1d640 1 -- 172.21.15.70:0/2597336187 >> v1:172.21.15.70:6816/32242 conn(0x7fcdfc04cb70 legacy=0x7fcdfc04f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:35.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:35.069+0000 7fce20b1d640 1 -- 172.21.15.70:0/2597336187 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fce1c068870 msgr2=0x7fce1c131720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:42:35.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:35.069+0000 7fce20b1d640 1 --2- 172.21.15.70:0/2597336187 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fce1c068870 0x7fce1c131720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fce1400cc20 tx=0x7fce14001a60 comp rx=0 tx=0).stop 2024-09-13T15:42:35.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:35.069+0000 7fce20b1d640 1 -- 172.21.15.70:0/2597336187 shutdown_connections 2024-09-13T15:42:35.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:35.069+0000 7fce20b1d640 1 --2- 172.21.15.70:0/2597336187 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fce1c098d30 0x7fce1c08d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:35.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:35.069+0000 7fce20b1d640 1 --2- 172.21.15.70:0/2597336187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce1c068cf0 0x7fce1c08cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:35.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:35.069+0000 7fce20b1d640 1 --2- 172.21.15.70:0/2597336187 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fce1c068870 0x7fce1c131720 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:42:35.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:35.069+0000 7fce20b1d640 1 -- 172.21.15.70:0/2597336187 >> 172.21.15.70:0/2597336187 conn(0x7fce1c09df70 msgr2=0x7fce1c093380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:42:35.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:35.069+0000 7fce20b1d640 1 -- 172.21.15.70:0/2597336187 shutdown_connections 2024-09-13T15:42:35.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:42:35.069+0000 7fce20b1d640 1 -- 172.21.15.70:0/2597336187 wait complete. 2024-09-13T15:42:35.080 INFO:teuthology.run_tasks:Running task install.upgrade... 2024-09-13T15:42:35.090 INFO:teuthology.task.install:project ceph config {'mon.a': None} overrides {'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'} 2024-09-13T15:42:35.090 INFO:teuthology.task.install:remote ubuntu@smithi070.front.sepia.ceph.com config {'flavor': 'default', 'sha1': '50bbe7f395997dcd871813949c49addbbf630361'} 2024-09-13T15:42:35.090 INFO:teuthology.task.install:extra packages: [] 2024-09-13T15:42:35.096 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T15:42:35.096 INFO:teuthology.task.install:Upgrading ceph rpm packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd 2024-09-13T15:42:35.096 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph-common --qf '%{VERSION}-%{RELEASE}' 2024-09-13T15:42:35.120 INFO:teuthology.orchestra.run.smithi070.stdout:17.2.7-1627.gbbdadfa3.el9 2024-09-13T15:42:35.120 INFO:teuthology.packaging:The installed version of ceph-common is 17.2.7-1627.gbbdadfa3.el9 2024-09-13T15:42:35.121 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T15:42:35.276 INFO:teuthology.task.install:Ceph rpm upgrade from 17.2.7-1627.gbbdadfa3.el9 to 18.2.4-850.g50bbe7f3 2024-09-13T15:42:35.276 INFO:teuthology.task.install.rpm:Host smithi070 is: centos 9.stream x86_64 2024-09-13T15:42:35.276 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T15:42:35.423 INFO:teuthology.task.install.rpm:Repo base URL: https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/ 2024-09-13T15:42:35.423 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/yum.repos.d/ceph.repo 2024-09-13T15:42:35.572 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.095512-reef/50bbe7f395997dcd871813949c49addbbf630361/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T15:42:35.572 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-13T15:42:35.572 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T15:42:35.598 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/50bbe7f395997dcd871813949c49addbbf630361/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T15:42:35.665 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T15:42:35.732 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T15:42:35.787 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2024-09-13T15:42:35.798 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2024-09-13T15:42:36.029 INFO:teuthology.orchestra.run.smithi070.stdout:60 files removed 2024-09-13T15:42:36.057 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y upgrade ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd 2024-09-13T15:42:36.612 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 293 kB/s | 77 kB 00:00 2024-09-13T15:42:36.896 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-13T15:42:37.172 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-13T15:42:38.972 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-13T15:42:42.057 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:01 2024-09-13T15:42:46.432 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-13T15:42:48.107 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-13T15:42:49.645 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-13T15:42:55.101 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-13T15:42:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:Package cephadm available, but not installed. 2024-09-13T15:42:57.365 INFO:teuthology.orchestra.run.smithi070.stdout:No match for argument: cephadm 2024-09-13T15:42:57.371 INFO:teuthology.orchestra.run.smithi070.stderr:Package ceph-mgr-dashboard available, but not installed. 2024-09-13T15:42:57.371 INFO:teuthology.orchestra.run.smithi070.stdout:No match for argument: ceph-mgr-dashboard 2024-09-13T15:42:57.371 INFO:teuthology.orchestra.run.smithi070.stderr:Package ceph-mgr-diskprediction-local available, but not installed. 2024-09-13T15:42:57.371 INFO:teuthology.orchestra.run.smithi070.stdout:No match for argument: ceph-mgr-diskprediction-local 2024-09-13T15:42:57.372 INFO:teuthology.orchestra.run.smithi070.stderr:Package ceph-mgr-rook available, but not installed. 2024-09-13T15:42:57.372 INFO:teuthology.orchestra.run.smithi070.stdout:No match for argument: ceph-mgr-rook 2024-09-13T15:42:57.372 INFO:teuthology.orchestra.run.smithi070.stderr:Package ceph-mgr-cephadm available, but not installed. 2024-09-13T15:42:57.373 INFO:teuthology.orchestra.run.smithi070.stdout:No match for argument: ceph-mgr-cephadm 2024-09-13T15:42:57.375 INFO:teuthology.orchestra.run.smithi070.stderr:Package ceph-volume available, but not installed. 2024-09-13T15:42:57.375 INFO:teuthology.orchestra.run.smithi070.stdout:No match for argument: ceph-volume 2024-09-13T15:42:57.463 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-13T15:42:57.466 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:42:57.466 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-13T15:42:57.466 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:42:57.466 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2024-09-13T15:42:57.466 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 6.5 k 2024-09-13T15:42:57.466 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 5.1 M 2024-09-13T15:42:57.466 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 18 M 2024-09-13T15:42:57.467 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 852 k 2024-09-13T15:42:57.467 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache 2024-09-13T15:42:57.467 INFO:teuthology.orchestra.run.smithi070.stdout: x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 143 k 2024-09-13T15:42:57.467 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 2.1 M 2024-09-13T15:42:57.467 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 1.5 M 2024-09-13T15:42:57.467 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:18.2.4-850.g50bbe7f3.el9 ceph-noarch 246 k 2024-09-13T15:42:57.467 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 4.7 M 2024-09-13T15:42:57.467 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 17 M 2024-09-13T15:42:57.467 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 7.8 M 2024-09-13T15:42:57.467 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 25 k 2024-09-13T15:42:57.467 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 40 M 2024-09-13T15:42:57.467 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 31 k 2024-09-13T15:42:57.467 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 712 k 2024-09-13T15:42:57.467 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 166 k 2024-09-13T15:42:57.468 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 127 k 2024-09-13T15:42:57.468 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 3.3 M 2024-09-13T15:42:57.468 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 477 k 2024-09-13T15:42:57.468 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 3.0 M 2024-09-13T15:42:57.468 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 4.5 M 2024-09-13T15:42:57.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 45 k 2024-09-13T15:42:57.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 129 k 2024-09-13T15:42:57.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 162 k 2024-09-13T15:42:57.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 319 k 2024-09-13T15:42:57.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 301 k 2024-09-13T15:42:57.468 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 99 k 2024-09-13T15:42:57.468 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 87 k 2024-09-13T15:42:57.468 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 3.0 M 2024-09-13T15:42:57.469 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:18.2.4-850.g50bbe7f3.el9 ceph 172 k 2024-09-13T15:42:57.469 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-13T15:42:57.469 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T15:42:57.469 INFO:teuthology.orchestra.run.smithi070.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T15:42:57.469 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T15:42:57.469 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T15:42:57.469 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T15:42:57.469 INFO:teuthology.orchestra.run.smithi070.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T15:42:57.469 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:42:57.470 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-13T15:42:57.470 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-13T15:42:57.470 INFO:teuthology.orchestra.run.smithi070.stdout:Install 6 Packages 2024-09-13T15:42:57.470 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 30 Packages 2024-09-13T15:42:57.470 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:42:57.470 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 119 M 2024-09-13T15:42:57.471 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-13T15:42:57.594 INFO:teuthology.orchestra.run.smithi070.stdout:(1/36): fmt-8.1.1-5.el9.x86_64.rpm 2.3 MB/s | 111 kB 00:00 2024-09-13T15:42:57.619 INFO:teuthology.orchestra.run.smithi070.stdout:(2/36): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-13T15:42:57.694 INFO:teuthology.orchestra.run.smithi070.stdout:(3/36): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-13T15:42:57.753 INFO:teuthology.orchestra.run.smithi070.stdout:(4/36): boost-program-options-1.75.0-8.el9.x86_ 519 kB/s | 107 kB 00:00 2024-09-13T15:42:57.778 INFO:teuthology.orchestra.run.smithi070.stdout:(5/36): re2-20211101-20.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-13T15:42:57.887 INFO:teuthology.orchestra.run.smithi070.stdout:(6/36): ceph-18.2.4-850.g50bbe7f3.el9.x86_64.rp 49 kB/s | 6.5 kB 00:00 2024-09-13T15:42:57.946 INFO:teuthology.orchestra.run.smithi070.stdout:(7/36): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-13T15:42:58.263 INFO:teuthology.orchestra.run.smithi070.stdout:(8/36): ceph-base-18.2.4-850.g50bbe7f3.el9.x86_ 11 MB/s | 5.1 MB 00:00 2024-09-13T15:42:58.322 INFO:teuthology.orchestra.run.smithi070.stdout:(9/36): ceph-immutable-object-cache-18.2.4-850. 2.4 MB/s | 143 kB 00:00 2024-09-13T15:42:58.497 INFO:teuthology.orchestra.run.smithi070.stdout:(10/36): ceph-fuse-18.2.4-850.g50bbe7f3.el9.x86 1.5 MB/s | 852 kB 00:00 2024-09-13T15:42:58.664 INFO:teuthology.orchestra.run.smithi070.stdout:(11/36): ceph-mds-18.2.4-850.g50bbe7f3.el9.x86_ 6.1 MB/s | 2.1 MB 00:00 2024-09-13T15:42:58.922 INFO:teuthology.orchestra.run.smithi070.stdout:(12/36): ceph-common-18.2.4-850.g50bbe7f3.el9.x 18 MB/s | 18 MB 00:01 2024-09-13T15:42:58.956 INFO:teuthology.orchestra.run.smithi070.stdout:(13/36): ceph-mgr-18.2.4-850.g50bbe7f3.el9.x86_ 3.2 MB/s | 1.5 MB 00:00 2024-09-13T15:42:59.031 INFO:teuthology.orchestra.run.smithi070.stdout:(14/36): ceph-mon-18.2.4-850.g50bbe7f3.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-13T15:42:59.081 INFO:teuthology.orchestra.run.smithi070.stdout:(15/36): ceph-selinux-18.2.4-850.g50bbe7f3.el9. 505 kB/s | 25 kB 00:00 2024-09-13T15:42:59.541 INFO:teuthology.orchestra.run.smithi070.stdout:(16/36): ceph-osd-18.2.4-850.g50bbe7f3.el9.x86_ 27 MB/s | 17 MB 00:00 2024-09-13T15:42:59.590 INFO:teuthology.orchestra.run.smithi070.stdout:(17/36): libcephfs-devel-18.2.4-850.g50bbe7f3.e 625 kB/s | 31 kB 00:00 2024-09-13T15:42:59.692 INFO:teuthology.orchestra.run.smithi070.stdout:(18/36): libcephfs2-18.2.4-850.g50bbe7f3.el9.x8 7.0 MB/s | 712 kB 00:00 2024-09-13T15:42:59.758 INFO:teuthology.orchestra.run.smithi070.stdout:(19/36): libcephsqlite-18.2.4-850.g50bbe7f3.el9 2.4 MB/s | 166 kB 00:00 2024-09-13T15:42:59.833 INFO:teuthology.orchestra.run.smithi070.stdout:(20/36): librados-devel-18.2.4-850.g50bbe7f3.el 1.7 MB/s | 127 kB 00:00 2024-09-13T15:43:00.066 INFO:teuthology.orchestra.run.smithi070.stdout:(21/36): librados2-18.2.4-850.g50bbe7f3.el9.x86 14 MB/s | 3.3 MB 00:00 2024-09-13T15:43:00.125 INFO:teuthology.orchestra.run.smithi070.stdout:(22/36): libradosstriper1-18.2.4-850.g50bbe7f3. 8.0 MB/s | 477 kB 00:00 2024-09-13T15:43:00.217 INFO:teuthology.orchestra.run.smithi070.stdout:(23/36): ceph-radosgw-18.2.4-850.g50bbe7f3.el9. 6.1 MB/s | 7.8 MB 00:01 2024-09-13T15:43:00.359 INFO:teuthology.orchestra.run.smithi070.stdout:(24/36): librbd1-18.2.4-850.g50bbe7f3.el9.x86_6 13 MB/s | 3.0 MB 00:00 2024-09-13T15:43:00.401 INFO:teuthology.orchestra.run.smithi070.stdout:(25/36): python3-ceph-argparse-18.2.4-850.g50bb 1.1 MB/s | 45 kB 00:00 2024-09-13T15:43:00.459 INFO:teuthology.orchestra.run.smithi070.stdout:(26/36): python3-ceph-common-18.2.4-850.g50bbe7 2.2 MB/s | 129 kB 00:00 2024-09-13T15:43:00.802 INFO:teuthology.orchestra.run.smithi070.stdout:(27/36): ceph-test-18.2.4-850.g50bbe7f3.el9.x86 23 MB/s | 40 MB 00:01 2024-09-13T15:43:00.826 INFO:teuthology.orchestra.run.smithi070.stdout:(28/36): python3-cephfs-18.2.4-850.g50bbe7f3.el 441 kB/s | 162 kB 00:00 2024-09-13T15:43:00.852 INFO:teuthology.orchestra.run.smithi070.stdout:(29/36): python3-rados-18.2.4-850.g50bbe7f3.el9 6.3 MB/s | 319 kB 00:00 2024-09-13T15:43:00.886 INFO:teuthology.orchestra.run.smithi070.stdout:(30/36): python3-rbd-18.2.4-850.g50bbe7f3.el9.x 5.0 MB/s | 301 kB 00:00 2024-09-13T15:43:00.911 INFO:teuthology.orchestra.run.smithi070.stdout:(31/36): python3-rgw-18.2.4-850.g50bbe7f3.el9.x 1.7 MB/s | 99 kB 00:00 2024-09-13T15:43:00.936 INFO:teuthology.orchestra.run.smithi070.stdout:(32/36): rbd-fuse-18.2.4-850.g50bbe7f3.el9.x86_ 1.7 MB/s | 87 kB 00:00 2024-09-13T15:43:01.004 INFO:teuthology.orchestra.run.smithi070.stdout:(33/36): librgw2-18.2.4-850.g50bbe7f3.el9.x86_6 5.7 MB/s | 4.5 MB 00:00 2024-09-13T15:43:01.029 INFO:teuthology.orchestra.run.smithi070.stdout:(34/36): rbd-nbd-18.2.4-850.g50bbe7f3.el9.x86_6 1.8 MB/s | 172 kB 00:00 2024-09-13T15:43:01.054 INFO:teuthology.orchestra.run.smithi070.stdout:(35/36): ceph-mgr-modules-core-18.2.4-850.g50bb 4.8 MB/s | 246 kB 00:00 2024-09-13T15:43:01.104 INFO:teuthology.orchestra.run.smithi070.stdout:(36/36): rbd-mirror-18.2.4-850.g50bbe7f3.el9.x8 16 MB/s | 3.0 MB 00:00 2024-09-13T15:43:01.109 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-13T15:43:01.109 INFO:teuthology.orchestra.run.smithi070.stdout:Total 33 MB/s | 119 MB 00:03 2024-09-13T15:43:01.122 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-13T15:43:01.178 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-13T15:43:01.178 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-13T15:43:02.021 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-13T15:43:02.021 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-13T15:43:04.649 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-13T15:43:04.738 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 1/66 2024-09-13T15:43:05.084 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 1/66 2024-09-13T15:43:05.157 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 2/66 2024-09-13T15:43:05.259 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 2/66 2024-09-13T15:43:05.315 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 3/66 2024-09-13T15:43:05.406 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 3/66 2024-09-13T15:43:05.472 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : python3-rados-2:18.2.4-850.g50bbe7f3.el9.x86_64 4/66 2024-09-13T15:43:05.509 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 5/66 2024-09-13T15:43:05.611 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 5/66 2024-09-13T15:43:05.652 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_64 6/66 2024-09-13T15:43:05.719 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_64 6/66 2024-09-13T15:43:05.770 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : python3-ceph-argparse-2:18.2.4-850.g50bbe7f3.el9.x 7/66 2024-09-13T15:43:05.841 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : python3-cephfs-2:18.2.4-850.g50bbe7f3.el9.x86_64 8/66 2024-09-13T15:43:05.898 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : python3-rbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 9/66 2024-09-13T15:43:06.049 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 10/66 2024-09-13T15:43:06.167 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : ceph-mgr-modules-core-2:18.2.4-850.g50bbe7f3.el9.n 11/66 2024-09-13T15:43:06.221 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : python3-ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86 12/66 2024-09-13T15:43:06.328 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : re2-1:20211101-20.el9.x86_64 13/66 2024-09-13T15:43:06.361 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 14/66 2024-09-13T15:43:06.388 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 15/66 2024-09-13T15:43:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fmt-8.1.1-5.el9.x86_64 16/66 2024-09-13T15:43:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 17/66 2024-09-13T15:43:06.952 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 18/66 2024-09-13T15:43:07.003 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 19/66 2024-09-13T15:43:07.075 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 19/66 2024-09-13T15:43:07.113 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : python3-rgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 20/66 2024-09-13T15:43:09.361 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 21/66 2024-09-13T15:43:09.441 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 21/66 2024-09-13T15:43:10.081 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 21/66 2024-09-13T15:43:10.253 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 22/66 2024-09-13T15:43:10.317 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 22/66 2024-09-13T15:43:10.353 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 23/66 2024-09-13T15:43:19.181 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 23/66 2024-09-13T15:43:19.276 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 24/66 2024-09-13T15:43:19.431 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 24/66 2024-09-13T15:43:19.480 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 25/66 2024-09-13T15:43:20.017 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 25/66 2024-09-13T15:43:20.070 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 26/66 2024-09-13T15:43:21.854 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 26/66 2024-09-13T15:43:21.898 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 27/66 2024-09-13T15:43:21.910 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 27/66 2024-09-13T15:43:22.792 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : ceph-2:18.2.4-850.g50bbe7f3.el9.x86_64 28/66 2024-09-13T15:43:22.841 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 29/66 2024-09-13T15:43:22.862 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 29/66 2024-09-13T15:43:22.908 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : ceph-immutable-object-cache-2:18.2.4-850.g50bbe7f3 30/66 2024-09-13T15:43:23.241 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-850.g50bbe7f3 30/66 2024-09-13T15:43:23.292 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 31/66 2024-09-13T15:43:27.885 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 31/66 2024-09-13T15:43:28.260 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : ceph-test-2:18.2.4-850.g50bbe7f3.el9.x86_64 32/66 2024-09-13T15:43:28.314 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : libcephfs-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 33/66 2024-09-13T15:43:28.363 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : rbd-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 34/66 2024-09-13T15:43:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : rbd-nbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 35/66 2024-09-13T15:43:28.541 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : ceph-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 36/66 2024-09-13T15:43:28.571 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : ceph-2:17.2.7-1627.gbbdadfa3.el9.x86_64 37/66 2024-09-13T15:43:28.603 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : ceph-test-2:17.2.7-1627.gbbdadfa3.el9.x86_64 38/66 2024-09-13T15:43:28.611 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 39/66 2024-09-13T15:43:28.611 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 39/66 2024-09-13T15:43:28.696 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 39/66 2024-09-13T15:43:28.721 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : rbd-nbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 40/66 2024-09-13T15:43:28.727 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 41/66 2024-09-13T15:43:28.727 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 41/66 2024-09-13T15:43:28.763 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 41/66 2024-09-13T15:43:28.769 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 42/66 2024-09-13T15:43:28.769 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 42/66 2024-09-13T15:43:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 42/66 2024-09-13T15:43:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : rbd-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 43/66 2024-09-13T15:43:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1627.gbbdadfa 44/66 2024-09-13T15:43:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : ceph-immutable-object-cache-2:17.2.7-1627.gbbdadfa 44/66 2024-09-13T15:43:28.873 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1627.gbbdadfa 44/66 2024-09-13T15:43:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : libcephfs-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 45/66 2024-09-13T15:43:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 46/66 2024-09-13T15:43:28.982 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 46/66 2024-09-13T15:43:29.007 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 47/66 2024-09-13T15:43:29.013 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 48/66 2024-09-13T15:43:29.013 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 48/66 2024-09-13T15:43:29.050 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 48/66 2024-09-13T15:43:29.055 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 49/66 2024-09-13T15:43:29.056 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 49/66 2024-09-13T15:43:29.092 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 49/66 2024-09-13T15:43:29.098 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 50/66 2024-09-13T15:43:29.098 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 50/66 2024-09-13T15:43:29.134 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 50/66 2024-09-13T15:43:29.134 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 51/66 2024-09-13T15:43:29.168 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 51/66 2024-09-13T15:43:29.174 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 52/66 2024-09-13T15:43:29.174 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 52/66 2024-09-13T15:43:29.271 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 52/66 2024-09-13T15:43:29.271 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 53/66 2024-09-13T15:43:29.305 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 53/66 2024-09-13T15:43:29.305 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86_6 54/66 2024-09-13T15:43:29.397 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86_6 54/66 2024-09-13T15:43:29.422 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : python3-cephfs-2:17.2.7-1627.gbbdadfa3.el9.x86_64 55/66 2024-09-13T15:43:29.448 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : python3-rbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 56/66 2024-09-13T15:43:29.448 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 57/66 2024-09-13T15:43:29.533 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 57/66 2024-09-13T15:43:29.557 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : python3-rgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 58/66 2024-09-13T15:43:29.558 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 59/66 2024-09-13T15:43:29.650 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 59/66 2024-09-13T15:43:29.675 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : python3-rados-2:17.2.7-1627.gbbdadfa3.el9.x86_64 60/66 2024-09-13T15:43:29.676 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 61/66 2024-09-13T15:43:29.760 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 61/66 2024-09-13T15:43:29.785 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : ceph-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 62/66 2024-09-13T15:43:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : python3-ceph-argparse-2:17.2.7-1627.gbbdadfa3.el9. 63/66 2024-09-13T15:43:29.836 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : python3-ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x8 64/66 2024-09-13T15:43:29.861 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : ceph-mgr-modules-core-2:17.2.7-1627.gbbdadfa3.el9. 65/66 2024-09-13T15:43:29.861 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 66/66 2024-09-13T15:43:31.042 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 66/66 2024-09-13T15:43:31.042 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 1/66 2024-09-13T15:43:31.042 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 2/66 2024-09-13T15:43:31.042 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 3/66 2024-09-13T15:43:31.042 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 4/66 2024-09-13T15:43:31.042 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 5/66 2024-09-13T15:43:31.042 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : re2-1:20211101-20.el9.x86_64 6/66 2024-09-13T15:43:31.042 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:18.2.4-850.g50bbe7f3.el9.x86_64 7/66 2024-09-13T15:43:31.043 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:17.2.7-1627.gbbdadfa3.el9.x86_64 8/66 2024-09-13T15:43:31.043 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 9/66 2024-09-13T15:43:31.043 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:17.2.7-1627.gbbdadfa3.el9.x86_64 10/66 2024-09-13T15:43:31.043 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 11/66 2024-09-13T15:43:31.043 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x86_64 12/66 2024-09-13T15:43:31.043 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 13/66 2024-09-13T15:43:31.043 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 14/66 2024-09-13T15:43:31.043 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-850.g50bbe7f3 15/66 2024-09-13T15:43:31.043 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1627.gbbdadfa 16/66 2024-09-13T15:43:31.043 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 17/66 2024-09-13T15:43:31.043 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:17.2.7-1627.gbbdadfa3.el9.x86_64 18/66 2024-09-13T15:43:31.043 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 19/66 2024-09-13T15:43:31.044 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:17.2.7-1627.gbbdadfa3.el9.x86_64 20/66 2024-09-13T15:43:31.044 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 21/66 2024-09-13T15:43:31.044 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:17.2.7-1627.gbbdadfa3.el9.x86_64 22/66 2024-09-13T15:43:31.044 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 23/66 2024-09-13T15:43:31.044 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 24/66 2024-09-13T15:43:31.044 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 25/66 2024-09-13T15:43:31.044 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 26/66 2024-09-13T15:43:31.044 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 27/66 2024-09-13T15:43:31.044 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:17.2.7-1627.gbbdadfa3.el9.x86_64 28/66 2024-09-13T15:43:31.044 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:18.2.4-850.g50bbe7f3.el9.x86_64 29/66 2024-09-13T15:43:31.044 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:17.2.7-1627.gbbdadfa3.el9.x86_64 30/66 2024-09-13T15:43:31.044 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 31/66 2024-09-13T15:43:31.047 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 32/66 2024-09-13T15:43:31.047 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 33/66 2024-09-13T15:43:31.048 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 34/66 2024-09-13T15:43:31.048 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 35/66 2024-09-13T15:43:31.048 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:17.2.7-1627.gbbdadfa3.el9.x86_64 36/66 2024-09-13T15:43:31.048 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 37/66 2024-09-13T15:43:31.048 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:17.2.7-1627.gbbdadfa3.el9.x86_64 38/66 2024-09-13T15:43:31.048 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 39/66 2024-09-13T15:43:31.048 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 40/66 2024-09-13T15:43:31.048 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_64 41/66 2024-09-13T15:43:31.048 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:17.2.7-1627.gbbdadfa3.el9.x86_6 42/66 2024-09-13T15:43:31.048 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 43/66 2024-09-13T15:43:31.048 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:17.2.7-1627.gbbdadfa3.el9.x86_64 44/66 2024-09-13T15:43:31.048 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 45/66 2024-09-13T15:43:31.049 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:17.2.7-1627.gbbdadfa3.el9.x86_64 46/66 2024-09-13T15:43:31.049 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:18.2.4-850.g50bbe7f3.el9.x 47/66 2024-09-13T15:43:31.049 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1627.gbbdadfa3.el9. 48/66 2024-09-13T15:43:31.049 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86 49/66 2024-09-13T15:43:31.049 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:17.2.7-1627.gbbdadfa3.el9.x8 50/66 2024-09-13T15:43:31.049 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:18.2.4-850.g50bbe7f3.el9.x86_64 51/66 2024-09-13T15:43:31.049 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:17.2.7-1627.gbbdadfa3.el9.x86_64 52/66 2024-09-13T15:43:31.049 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:18.2.4-850.g50bbe7f3.el9.x86_64 53/66 2024-09-13T15:43:31.049 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:17.2.7-1627.gbbdadfa3.el9.x86_64 54/66 2024-09-13T15:43:31.049 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 55/66 2024-09-13T15:43:31.049 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 56/66 2024-09-13T15:43:31.049 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 57/66 2024-09-13T15:43:31.049 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:17.2.7-1627.gbbdadfa3.el9.x86_64 58/66 2024-09-13T15:43:31.049 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 59/66 2024-09-13T15:43:31.050 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:17.2.7-1627.gbbdadfa3.el9.x86_64 60/66 2024-09-13T15:43:31.050 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 61/66 2024-09-13T15:43:31.050 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:17.2.7-1627.gbbdadfa3.el9.x86_64 62/66 2024-09-13T15:43:31.050 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 63/66 2024-09-13T15:43:31.050 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:17.2.7-1627.gbbdadfa3.el9.x86_64 64/66 2024-09-13T15:43:31.050 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-850.g50bbe7f3.el9.n 65/66 2024-09-13T15:43:32.116 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1627.gbbdadfa3.el9. 66/66 2024-09-13T15:43:32.116 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:43:32.116 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2024-09-13T15:43:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:18.2.4-850.g50bbe7f3.el9.noarch 2024-09-13T15:43:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:18.2.4-850.g50bbe7f3.el9.x86_64 2024-09-13T15:43:32.119 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-13T15:43:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options-1.75.0-8.el9.x86_64 fmt-8.1.1-5.el9.x86_64 2024-09-13T15:43:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-9.0.0-11.el9.x86_64 libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T15:43:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs-9.0.0-11.el9.x86_64 re2-1:20211101-20.el9.x86_64 2024-09-13T15:43:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:43:32.119 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-13T15:43:32.495 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=50bbe7f395997dcd871813949c49addbbf630361 2024-09-13T15:43:32.646 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T15:43:32.668 INFO:teuthology.orchestra.run.smithi070.stdout:18.2.4-850.g50bbe7f3.el9 2024-09-13T15:43:32.668 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-850.g50bbe7f3.el9 2024-09-13T15:43:32.668 INFO:teuthology.task.install:The correct ceph version 18.2.4-850.g50bbe7f3 is installed. 2024-09-13T15:43:32.670 INFO:teuthology.run_tasks:Running task print... 2024-09-13T15:43:32.678 INFO:teuthology.task.print:**** done install.upgrade 2024-09-13T15:43:32.678 INFO:teuthology.run_tasks:Running task ceph.restart... 2024-09-13T15:43:32.689 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T15:43:32.689 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:43:32.719 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T15:43:32.720 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T15:43:32.720 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T15:43:32.762 INFO:tasks.ceph.mon.a:Started 2024-09-13T15:43:32.763 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T15:43:32.763 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:43:32.774 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T15:43:32.775 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T15:43:32.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T15:43:32.817 INFO:tasks.ceph.mon.b:Started 2024-09-13T15:43:32.817 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T15:43:32.818 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:43:32.830 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T15:43:32.830 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T15:43:32.830 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T15:43:32.872 INFO:tasks.ceph.mon.c:Started 2024-09-13T15:43:32.872 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T15:43:32.873 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:43:32.896 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T15:43:32.896 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T15:43:32.897 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T15:43:32.939 INFO:tasks.ceph.mgr.x:Started 2024-09-13T15:43:32.939 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T15:43:32.940 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:43:32.964 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T15:43:32.964 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T15:43:32.965 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T15:43:33.008 INFO:tasks.ceph.mgr.y:Started 2024-09-13T15:43:33.008 INFO:teuthology.run_tasks:Running task ceph.healthy... 2024-09-13T15:43:33.018 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T15:43:33.018 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T15:43:33.018 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T15:43:33.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d7a053640 1 Processor -- start 2024-09-13T15:43:33.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d7a053640 1 -- start start 2024-09-13T15:43:33.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d7a053640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d74132710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:33.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d7a053640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d74135040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:33.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d7a053640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d74097940 con 0x7f9d7406bb10 2024-09-13T15:43:33.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d7a053640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d74097500 con 0x7f9d74132c50 2024-09-13T15:43:33.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d7a053640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7406d100 con 0x7f9d7406c600 2024-09-13T15:43:33.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d72ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d74132710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:33.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d73fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d74135040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:33.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d72ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d74132710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:35968/0 (socket says 172.21.15.70:35968) 2024-09-13T15:43:33.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d737fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9d7406bb10 0x7f9d7406bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44062/0 (socket says 172.21.15.70:44062) 2024-09-13T15:43:33.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3581273796 learned_addr learned my addr 172.21.15.70:0/3581273796 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:33.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d73fff640 1 -- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 msgr2=0x7f9d74135040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T15:43:33.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d73fff640 1 -- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 msgr2=0x7f9d74135040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:43:33.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d73fff640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d74135040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T15:43:33.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 msgr2=0x7f9d74132710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:43:33.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 msgr2=0x7f9d74132710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:43:33.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d72ffd640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d74132710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T15:43:33.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d73fff640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d74135040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:43:33.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.117+0000 7f9d72ffd640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d74132710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:43:33.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.318+0000 7f9d73fff640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d74135040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:33.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.318+0000 7f9d72ffd640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d74132710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:33.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.318+0000 7f9d73fff640 1 -- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 msgr2=0x7f9d74135040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:43:33.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.318+0000 7f9d73fff640 1 -- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 msgr2=0x7f9d74135040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:43:33.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.318+0000 7f9d73fff640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d74135040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T15:43:33.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.318+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 msgr2=0x7f9d74132710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T15:43:33.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.318+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 msgr2=0x7f9d74132710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:43:33.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.318+0000 7f9d72ffd640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d74132710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T15:43:33.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.318+0000 7f9d73fff640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d74135040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T15:43:33.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.318+0000 7f9d72ffd640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d74132710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T15:43:33.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.719+0000 7f9d72ffd640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d74132710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:33.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.719+0000 7f9d73fff640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d74135040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:33.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.719+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 msgr2=0x7f9d74132710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T15:43:33.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.719+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 msgr2=0x7f9d74132710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:43:33.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.719+0000 7f9d72ffd640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d74132710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T15:43:33.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.719+0000 7f9d73fff640 1 -- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 msgr2=0x7f9d74135040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:43:33.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.719+0000 7f9d73fff640 1 -- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 msgr2=0x7f9d74135040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:43:33.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.719+0000 7f9d73fff640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d74135040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T15:43:33.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.719+0000 7f9d72ffd640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d74132710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T15:43:33.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:33.719+0000 7f9d73fff640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d74135040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T15:43:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:34.520+0000 7f9d72ffd640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d74132710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:34.520+0000 7f9d73fff640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d74135040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:34.520+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 msgr2=0x7f9d74132710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:43:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:34.520+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 msgr2=0x7f9d74132710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:43:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:34.520+0000 7f9d72ffd640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d74132710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T15:43:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:34.520+0000 7f9d72ffd640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d74132710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T15:43:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:34.520+0000 7f9d73fff640 1 -- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 msgr2=0x7f9d74135040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T15:43:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:34.520+0000 7f9d73fff640 1 -- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 msgr2=0x7f9d74135040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:43:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:34.520+0000 7f9d73fff640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d74135040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T15:43:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:34.520+0000 7f9d73fff640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d74135040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T15:43:36.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.117+0000 7f9d717fa640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 msgr2=0x7f9d74132710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T15:43:36.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.117+0000 7f9d717fa640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d74132710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:36.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.117+0000 7f9d717fa640 1 -- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 msgr2=0x7f9d74135040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T15:43:36.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.117+0000 7f9d717fa640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d74135040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:36.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.117+0000 7f9d717fa640 1 -- 172.21.15.70:0/3581273796 >> v1:172.21.15.70:6789/0 conn(0x7f9d7406bb10 legacy=0x7f9d7406bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:36.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.117+0000 7f9d717fa640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 0x7f9d54005ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:36.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.117+0000 7f9d717fa640 1 -- 172.21.15.70:0/3581273796 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74097500 con 0x7f9d54003650 2024-09-13T15:43:36.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.117+0000 7f9d737fe640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 0x7f9d54005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:36.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.117+0000 7f9d737fe640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 msgr2=0x7f9d54005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:43:36.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.117+0000 7f9d737fe640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 msgr2=0x7f9d54005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:43:36.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.117+0000 7f9d737fe640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 0x7f9d54005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T15:43:36.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.117+0000 7f9d737fe640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 0x7f9d54005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T15:43:36.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.318+0000 7f9d737fe640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 0x7f9d54005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:36.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.318+0000 7f9d737fe640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 msgr2=0x7f9d54005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:43:36.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.318+0000 7f9d737fe640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 msgr2=0x7f9d54005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:43:36.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.318+0000 7f9d737fe640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 0x7f9d54005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T15:43:36.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.318+0000 7f9d737fe640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 0x7f9d54005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T15:43:36.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.719+0000 7f9d737fe640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 0x7f9d54005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:36.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.719+0000 7f9d737fe640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 msgr2=0x7f9d54005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:43:36.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.719+0000 7f9d737fe640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 msgr2=0x7f9d54005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:43:36.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.719+0000 7f9d737fe640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 0x7f9d54005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T15:43:36.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:36.719+0000 7f9d737fe640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 0x7f9d54005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T15:43:37.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:37.520+0000 7f9d737fe640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 0x7f9d54005ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:37.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:37.520+0000 7f9d737fe640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 msgr2=0x7f9d54005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T15:43:37.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:37.520+0000 7f9d737fe640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 msgr2=0x7f9d54005ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T15:43:37.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:37.520+0000 7f9d737fe640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 0x7f9d54005ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T15:43:37.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:37.520+0000 7f9d737fe640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 0x7f9d54005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T15:43:39.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.117+0000 7f9d717fa640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 msgr2=0x7f9d54005ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T15:43:39.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.117+0000 7f9d717fa640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 0x7f9d54005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.117+0000 7f9d717fa640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d54008350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.117+0000 7f9d717fa640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d5400e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.117+0000 7f9d717fa640 1 -- 172.21.15.70:0/3581273796 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d7406da40 con 0x7f9d7406bb10 2024-09-13T15:43:39.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.117+0000 7f9d717fa640 1 -- 172.21.15.70:0/3581273796 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d74097500 con 0x7f9d74132c50 2024-09-13T15:43:39.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.117+0000 7f9d717fa640 1 -- 172.21.15.70:0/3581273796 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7406d100 con 0x7f9d7406c600 2024-09-13T15:43:39.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.117+0000 7f9d73fff640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d5400e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.118+0000 7f9d737fe640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d54008350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.118+0000 7f9d727fc640 1 -- 172.21.15.70:0/3581273796 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330616937 0 0) 0x7f9d7406da40 con 0x7f9d7406bb10 2024-09-13T15:43:39.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.118+0000 7f9d73fff640 1 -- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 msgr2=0x7f9d54008350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:39.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.118+0000 7f9d73fff640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d54008350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.118+0000 7f9d73fff640 1 -- 172.21.15.70:0/3581273796 >> v1:172.21.15.70:6789/0 conn(0x7f9d7406bb10 legacy=0x7f9d5400a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:39.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.118+0000 7f9d73fff640 1 -- 172.21.15.70:0/3581273796 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5400f6a0 con 0x7f9d7406c600 2024-09-13T15:43:39.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.119+0000 7f9d73fff640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d5400e090 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009610 tx=0x7f9d6c02b270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92a251ec72da15d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:39.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.119+0000 7f9d727fc640 1 -- 172.21.15.70:0/3581273796 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c02bd80 con 0x7f9d7406c600 2024-09-13T15:43:39.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.119+0000 7f9d727fc640 1 -- 172.21.15.70:0/3581273796 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c02f070 con 0x7f9d7406c600 2024-09-13T15:43:39.122 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.119+0000 7f9d727fc640 1 -- 172.21.15.70:0/3581273796 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c035640 con 0x7f9d7406c600 2024-09-13T15:43:39.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.120+0000 7f9d7a053640 1 -- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 msgr2=0x7f9d5400e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:39.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.120+0000 7f9d7a053640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d5400e090 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009610 tx=0x7f9d6c02b270 comp rx=0 tx=0).stop 2024-09-13T15:43:39.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.120+0000 7f9d7a053640 1 -- 172.21.15.70:0/3581273796 shutdown_connections 2024-09-13T15:43:39.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.120+0000 7f9d7a053640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d7406c600 0x7f9d5400e090 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.120+0000 7f9d7a053640 1 --2- 172.21.15.70:0/3581273796 >> v2:172.21.15.70:3300/0 conn(0x7f9d74132c50 0x7f9d54008350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.120+0000 7f9d7a053640 1 --2- 172.21.15.70:0/3581273796 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d54003650 0x7f9d54005ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.120+0000 7f9d7a053640 1 -- 172.21.15.70:0/3581273796 >> 172.21.15.70:0/3581273796 conn(0x7f9d7409fe20 msgr2=0x7f9d740a0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:39.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.120+0000 7f9d7a053640 1 -- 172.21.15.70:0/3581273796 shutdown_connections 2024-09-13T15:43:39.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.120+0000 7f9d7a053640 1 -- 172.21.15.70:0/3581273796 wait complete. 2024-09-13T15:43:39.125 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:39.120+0000 7fd9b5c49040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T15:43:39.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.121+0000 7f9d7a053640 1 Processor -- start 2024-09-13T15:43:39.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.121+0000 7f9d7a053640 1 -- start start 2024-09-13T15:43:39.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.121+0000 7f9d7a053640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d54003650 0x7f9d7412eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.121+0000 7f9d7a053640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d7406bb10 0x7f9d7412f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.121+0000 7f9d7a053640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d7406c600 0x7f9d741c21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.121+0000 7f9d7a053640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74131190 con 0x7f9d7406bb10 2024-09-13T15:43:39.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.121+0000 7f9d7a053640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d74131d30 con 0x7f9d54003650 2024-09-13T15:43:39.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.121+0000 7f9d7a053640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d741315c0 con 0x7f9d7406c600 2024-09-13T15:43:39.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.121+0000 7f9d72ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d7406bb10 0x7f9d7412f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d73fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d7406c600 0x7f9d741c21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d72ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d7406bb10 0x7f9d7412f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37106/0 (socket says 172.21.15.70:37106) 2024-09-13T15:43:39.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d72ffd640 1 -- 172.21.15.70:0/967156037 learned_addr learned my addr 172.21.15.70:0/967156037 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:39.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d737fe640 1 --2- 172.21.15.70:0/967156037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d54003650 0x7f9d7412eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d72ffd640 1 -- 172.21.15.70:0/967156037 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d7406c600 msgr2=0x7f9d741c21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:39.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d72ffd640 1 --2- 172.21.15.70:0/967156037 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d7406c600 0x7f9d741c21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d72ffd640 1 -- 172.21.15.70:0/967156037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d54003650 msgr2=0x7f9d7412eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:39.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d72ffd640 1 --2- 172.21.15.70:0/967156037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d54003650 0x7f9d7412eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d72ffd640 1 -- 172.21.15.70:0/967156037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c003e70 con 0x7f9d7406bb10 2024-09-13T15:43:39.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d737fe640 1 --2- 172.21.15.70:0/967156037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d54003650 0x7f9d7412eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:39.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d72ffd640 1 --2- 172.21.15.70:0/967156037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d7406bb10 0x7f9d7412f030 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9d64012550 tx=0x7f9d64012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:39.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d6bfff640 1 -- 172.21.15.70:0/967156037 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64014820 con 0x7f9d7406bb10 2024-09-13T15:43:39.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d6bfff640 1 -- 172.21.15.70:0/967156037 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6400c040 con 0x7f9d7406bb10 2024-09-13T15:43:39.129 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d7a053640 1 -- 172.21.15.70:0/967156037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d74130400 con 0x7f9d7406bb10 2024-09-13T15:43:39.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d6bfff640 1 -- 172.21.15.70:0/967156037 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64014be0 con 0x7f9d7406bb10 2024-09-13T15:43:39.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.122+0000 7f9d7a053640 1 -- 172.21.15.70:0/967156037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7412f570 con 0x7f9d7406bb10 2024-09-13T15:43:39.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.123+0000 7f9d6bfff640 1 -- 172.21.15.70:0/967156037 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f9d64014d40 con 0x7f9d7406bb10 2024-09-13T15:43:39.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.124+0000 7f9d737fe640 1 -- 172.21.15.70:0/967156037 >> v1:172.21.15.70:6816/32242 conn(0x7f9d5404d950 legacy=0x7f9d5404fe10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6816/32242 2024-09-13T15:43:39.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.124+0000 7f9d6bfff640 1 -- 172.21.15.70:0/967156037 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f9d64063420 con 0x7f9d7406bb10 2024-09-13T15:43:39.130 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.124+0000 7f9d7a053640 1 -- 172.21.15.70:0/967156037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d740a04f0 con 0x7f9d7406bb10 2024-09-13T15:43:39.131 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.128+0000 7f9d6bfff640 1 -- 172.21.15.70:0/967156037 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f9d6401c080 con 0x7f9d7406bb10 2024-09-13T15:43:39.162 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:39.161+0000 7f1d6571d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T15:43:39.165 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:39.164+0000 7fd9b5c49040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T15:43:39.205 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:39.203+0000 7f1d6571d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T15:43:39.217 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:39.216+0000 7fd9b5c49040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T15:43:39.254 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:39.253+0000 7f1d6571d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T15:43:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.293+0000 7f9d7a053640 1 -- 172.21.15.70:0/967156037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f9d7412d3f0 con 0x7f9d7406bb10 2024-09-13T15:43:39.296 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.295+0000 7f9d6bfff640 1 -- 172.21.15.70:0/967156037 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v6) v1 ==== 74+0+116310 (secure 0 0 0) 0x7f9d6403e0b0 con 0x7f9d7406bb10 2024-09-13T15:43:39.297 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:43:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.297+0000 7f9d7a053640 1 -- 172.21.15.70:0/967156037 >> v1:172.21.15.70:6816/32242 conn(0x7f9d5404d950 legacy=0x7f9d5404fe10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:43:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.297+0000 7f9d7a053640 1 -- 172.21.15.70:0/967156037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d7406bb10 msgr2=0x7f9d7412f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.297+0000 7f9d7a053640 1 --2- 172.21.15.70:0/967156037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d7406bb10 0x7f9d7412f030 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9d64012550 tx=0x7f9d64012a10 comp rx=0 tx=0).stop 2024-09-13T15:43:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.297+0000 7f9d7a053640 1 -- 172.21.15.70:0/967156037 shutdown_connections 2024-09-13T15:43:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.297+0000 7f9d7a053640 1 --2- 172.21.15.70:0/967156037 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9d7406c600 0x7f9d741c21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.297+0000 7f9d7a053640 1 --2- 172.21.15.70:0/967156037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d7406bb10 0x7f9d7412f030 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.297+0000 7f9d7a053640 1 --2- 172.21.15.70:0/967156037 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9d54003650 0x7f9d7412eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.297+0000 7f9d7a053640 1 -- 172.21.15.70:0/967156037 >> 172.21.15.70:0/967156037 conn(0x7f9d7409fe20 msgr2=0x7f9d74129f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.297+0000 7f9d7a053640 1 -- 172.21.15.70:0/967156037 shutdown_connections 2024-09-13T15:43:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.297+0000 7f9d7a053640 1 -- 172.21.15.70:0/967156037 wait complete. 2024-09-13T15:43:39.309 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":6,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6816","nonce":32242}]},"active_addr":"172.21.15.70:6816/32242","active_change":"2024-09-13T15:42:12.965910+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":3873928986}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":2914106548}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":3345804336}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":3671770748}]}]} 2024-09-13T15:43:39.312 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T15:43:39.312 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T15:43:39.313 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T15:43:39.341 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:39.340+0000 7fd9b5c49040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T15:43:39.356 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:39.355+0000 7f1d6571d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T15:43:39.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.411+0000 7f6c6ad9c640 1 Processor -- start 2024-09-13T15:43:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.411+0000 7f6c6ad9c640 1 -- start start 2024-09-13T15:43:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.412+0000 7f6c6ad9c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6c6406bb10 0x7f6c6406bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.412+0000 7f6c6ad9c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c6412db00 0x7f6c6412df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.412+0000 7f6c6ad9c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c64097940 con 0x7f6c6406c430 2024-09-13T15:43:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.412+0000 7f6c6ad9c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c64097500 con 0x7f6c6412db00 2024-09-13T15:43:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.412+0000 7f6c6ad9c640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6406c810 con 0x7f6c6406bb10 2024-09-13T15:43:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.412+0000 7f6c69339640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c6412db00 0x7f6c6412df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.412+0000 7f6c63fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6c6406c430 0x7f6c6412d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58096/0 (socket says 172.21.15.70:58096) 2024-09-13T15:43:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.412+0000 7f6c63fff640 1 -- 172.21.15.70:0/86976331 learned_addr learned my addr 172.21.15.70:0/86976331 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.412+0000 7f6c68b38640 1 --2- 172.21.15.70:0/86976331 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6c6406bb10 0x7f6c6406bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.412+0000 7f6c637fe640 1 -- 172.21.15.70:0/86976331 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420008308 0 0) 0x7f6c64097940 con 0x7f6c6406c430 2024-09-13T15:43:39.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.412+0000 7f6c637fe640 1 -- 172.21.15.70:0/86976331 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c54003520 con 0x7f6c6406c430 2024-09-13T15:43:39.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.412+0000 7f6c69339640 1 -- 172.21.15.70:0/86976331 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6c6406bb10 msgr2=0x7f6c6406bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:39.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.412+0000 7f6c69339640 1 --2- 172.21.15.70:0/86976331 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6c6406bb10 0x7f6c6406bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.412+0000 7f6c69339640 1 -- 172.21.15.70:0/86976331 >> v1:172.21.15.70:6789/0 conn(0x7f6c6406c430 legacy=0x7f6c6412d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:39.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.412+0000 7f6c69339640 1 -- 172.21.15.70:0/86976331 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6406d100 con 0x7f6c6412db00 2024-09-13T15:43:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.413+0000 7f6c69339640 1 --2- 172.21.15.70:0/86976331 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c6412db00 0x7f6c6412df70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009dc0 tx=0x7f6c5c02a520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aafa7d299a9e635 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.413+0000 7f6c637fe640 1 -- 172.21.15.70:0/86976331 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c007740 con 0x7f6c6412db00 2024-09-13T15:43:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.413+0000 7f6c637fe640 1 -- 172.21.15.70:0/86976331 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c02d050 con 0x7f6c6412db00 2024-09-13T15:43:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.413+0000 7f6c637fe640 1 -- 172.21.15.70:0/86976331 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c007aa0 con 0x7f6c6412db00 2024-09-13T15:43:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.414+0000 7f6c6ad9c640 1 -- 172.21.15.70:0/86976331 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c6412db00 msgr2=0x7f6c6412df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.414+0000 7f6c6ad9c640 1 --2- 172.21.15.70:0/86976331 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c6412db00 0x7f6c6412df70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009dc0 tx=0x7f6c5c02a520 comp rx=0 tx=0).stop 2024-09-13T15:43:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.414+0000 7f6c6ad9c640 1 -- 172.21.15.70:0/86976331 shutdown_connections 2024-09-13T15:43:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.414+0000 7f6c6ad9c640 1 --2- 172.21.15.70:0/86976331 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c6412db00 0x7f6c6412df70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.414+0000 7f6c6ad9c640 1 --2- 172.21.15.70:0/86976331 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6c6406bb10 0x7f6c6406bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.414+0000 7f6c6ad9c640 1 -- 172.21.15.70:0/86976331 >> 172.21.15.70:0/86976331 conn(0x7f6c6409fe20 msgr2=0x7f6c640a0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.414+0000 7f6c6ad9c640 1 -- 172.21.15.70:0/86976331 shutdown_connections 2024-09-13T15:43:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.414+0000 7f6c6ad9c640 1 -- 172.21.15.70:0/86976331 wait complete. 2024-09-13T15:43:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.415+0000 7f6c6ad9c640 1 Processor -- start 2024-09-13T15:43:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.415+0000 7f6c6ad9c640 1 -- start start 2024-09-13T15:43:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.415+0000 7f6c6ad9c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c6406bb10 0x7f6c6412ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.415+0000 7f6c6ad9c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6c6406c430 0x7f6c6412f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.415+0000 7f6c6ad9c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c6412db00 0x7f6c6412f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.415+0000 7f6c6ad9c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c64131500 con 0x7f6c6412db00 2024-09-13T15:43:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.415+0000 7f6c6ad9c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c64132120 con 0x7f6c6406bb10 2024-09-13T15:43:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.415+0000 7f6c6ad9c640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c64134ea0 con 0x7f6c6406c430 2024-09-13T15:43:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.415+0000 7f6c63fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6c6406c430 0x7f6c6412f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c69339640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c6412db00 0x7f6c6412f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.415+0000 7f6c68b38640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c6406bb10 0x7f6c6412ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c69339640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c6412db00 0x7f6c6412f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37122/0 (socket says 172.21.15.70:37122) 2024-09-13T15:43:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c69339640 1 -- 172.21.15.70:0/2260320245 learned_addr learned my addr 172.21.15.70:0/2260320245 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c63fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6c6406c430 0x7f6c6412f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:37832/0 (socket says 172.21.15.70:37832) 2024-09-13T15:43:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c63fff640 1 -- 172.21.15.70:0/2260320245 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c6406bb10 msgr2=0x7f6c6412ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c63fff640 1 --2- 172.21.15.70:0/2260320245 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c6406bb10 0x7f6c6412ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c63fff640 1 -- 172.21.15.70:0/2260320245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c6412db00 msgr2=0x7f6c6412f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c63fff640 1 --2- 172.21.15.70:0/2260320245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c6412db00 0x7f6c6412f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c63fff640 1 -- 172.21.15.70:0/2260320245 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c009a70 con 0x7f6c6406c430 2024-09-13T15:43:39.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c69339640 1 --2- 172.21.15.70:0/2260320245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c6412db00 0x7f6c6412f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:39.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c68b38640 1 --2- 172.21.15.70:0/2260320245 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c6406bb10 0x7f6c6412ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:39.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c63fff640 1 --2- 172.21.15.70:0/2260320245 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6c6406c430 0x7f6c6412f150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000c460 tx=0x7f6c5000c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:39.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c617fa640 1 -- 172.21.15.70:0/2260320245 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50005ce0 con 0x7f6c6406c430 2024-09-13T15:43:39.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c617fa640 1 -- 172.21.15.70:0/2260320245 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c50005e40 con 0x7f6c6406c430 2024-09-13T15:43:39.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c6ad9c640 1 -- 172.21.15.70:0/2260320245 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c641316c0 con 0x7f6c6406c430 2024-09-13T15:43:39.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.416+0000 7f6c617fa640 1 -- 172.21.15.70:0/2260320245 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50007270 con 0x7f6c6406c430 2024-09-13T15:43:39.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.417+0000 7f6c6ad9c640 1 -- 172.21.15.70:0/2260320245 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c64127d40 con 0x7f6c6406c430 2024-09-13T15:43:39.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.418+0000 7f6c6ad9c640 1 -- 172.21.15.70:0/2260320245 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c64099460 con 0x7f6c6406c430 2024-09-13T15:43:39.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.418+0000 7f6c617fa640 1 -- 172.21.15.70:0/2260320245 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f6c500073d0 con 0x7f6c6406c430 2024-09-13T15:43:39.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.418+0000 7f6c617fa640 1 -- 172.21.15.70:0/2260320245 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f6c50062820 con 0x7f6c6406c430 2024-09-13T15:43:39.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.418+0000 7f6c68b38640 1 -- 172.21.15.70:0/2260320245 >> v1:172.21.15.70:6816/32242 conn(0x7f6c4804cfc0 legacy=0x7f6c4804f480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6816/32242 2024-09-13T15:43:39.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.422+0000 7f6c617fa640 1 -- 172.21.15.70:0/2260320245 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f6c5003d680 con 0x7f6c6406c430 2024-09-13T15:43:39.458 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:39.456+0000 7fd9b5c49040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T15:43:39.458 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:39.457+0000 7f1d6571d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T15:43:39.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.560+0000 7f6c6ad9c640 1 -- 172.21.15.70:0/2260320245 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c6412c3c0 con 0x7f6c6406c430 2024-09-13T15:43:39.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.560+0000 7f6c617fa640 1 -- 172.21.15.70:0/2260320245 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+8596 (secure 0 0 0) 0x7f6c50017020 con 0x7f6c6406c430 2024-09-13T15:43:39.562 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:43:39.562 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":18,"fsid":"45bbcdad-f4fb-4639-ab42-ccda8de151dd","created":"2024-09-13T15:42:09.863982+0000","modified":"2024-09-13T15:42:34.654949+0000","last_up_change":"2024-09-13T15:42:20.070768+0000","last_in_change":"2024-09-13T15:42:11.714506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T15:42:21.000499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T15:42:26.149909+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"265c6f1d-08a7-4c43-b2a6-2417ab814825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":32759}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":32759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":32759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":32759}]},"public_addr":"172.21.15.70:6804/32759","cluster_addr":"172.21.15.70:6805/32759","heartbeat_back_addr":"172.21.15.70:6807/32759","heartbeat_front_addr":"172.21.15.70:6806/32759","state":["exists","up"]},{"osd":1,"uuid":"ba4d90f9-6972-45df-800f-92ad808b06ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":32758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":32758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":32758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":32758}]},"public_addr":"172.21.15.70:6800/32758","cluster_addr":"172.21.15.70:6801/32758","heartbeat_back_addr":"172.21.15.70:6803/32758","heartbeat_front_addr":"172.21.15.70:6802/32758","state":["exists","up"]},{"osd":2,"uuid":"b74d8eb5-3082-4d69-a755-7f5f92dec5b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":32760}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":32760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":32760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":32760}]},"public_addr":"172.21.15.70:6812/32760","cluster_addr":"172.21.15.70:6813/32760","heartbeat_back_addr":"172.21.15.70:6815/32760","heartbeat_front_addr":"172.21.15.70:6814/32760","state":["exists","up"]},{"osd":3,"uuid":"02835b7d-e2cf-4ef8-87ba-b225593689d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":32769}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":32769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":32769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":32769}]},"public_addr":"172.21.15.70:6808/32769","cluster_addr":"172.21.15.70:6809/32769","heartbeat_back_addr":"172.21.15.70:6811/32769","heartbeat_front_addr":"172.21.15.70:6810/32769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.853420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.807491+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.925567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.883852+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T15:43:39.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.562+0000 7f6c6ad9c640 1 -- 172.21.15.70:0/2260320245 >> v1:172.21.15.70:6816/32242 conn(0x7f6c4804cfc0 legacy=0x7f6c4804f480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:43:39.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.562+0000 7f6c6ad9c640 1 -- 172.21.15.70:0/2260320245 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6c6406c430 msgr2=0x7f6c6412f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:39.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.562+0000 7f6c6ad9c640 1 --2- 172.21.15.70:0/2260320245 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6c6406c430 0x7f6c6412f150 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000c460 tx=0x7f6c5000c920 comp rx=0 tx=0).stop 2024-09-13T15:43:39.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.562+0000 7f6c6ad9c640 1 -- 172.21.15.70:0/2260320245 shutdown_connections 2024-09-13T15:43:39.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.562+0000 7f6c6ad9c640 1 --2- 172.21.15.70:0/2260320245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c6412db00 0x7f6c6412f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.562+0000 7f6c6ad9c640 1 --2- 172.21.15.70:0/2260320245 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6c6406c430 0x7f6c6412f150 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.562+0000 7f6c6ad9c640 1 --2- 172.21.15.70:0/2260320245 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6c6406bb10 0x7f6c6412ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.562+0000 7f6c6ad9c640 1 -- 172.21.15.70:0/2260320245 >> 172.21.15.70:0/2260320245 conn(0x7f6c6409fe20 msgr2=0x7f6c641272e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:39.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.562+0000 7f6c6ad9c640 1 -- 172.21.15.70:0/2260320245 shutdown_connections 2024-09-13T15:43:39.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.562+0000 7f6c6ad9c640 1 -- 172.21.15.70:0/2260320245 wait complete. 2024-09-13T15:43:39.575 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T15:43:39.575 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T15:43:39.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.714+0000 7f9fece4f640 1 Processor -- start 2024-09-13T15:43:39.715 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.714+0000 7f9fece4f640 1 -- start start 2024-09-13T15:43:39.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.714+0000 7f9fece4f640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9fe806bb10 0x7f9fe806bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.714+0000 7f9fece4f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9fe806c430 0x7f9fe812d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.714+0000 7f9fece4f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fe812d930 0x7f9fe812dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.714+0000 7f9fece4f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe8097940 con 0x7f9fe806c430 2024-09-13T15:43:39.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.714+0000 7f9fece4f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe812e2e0 con 0x7f9fe812d930 2024-09-13T15:43:39.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.714+0000 7f9fece4f640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8097500 con 0x7f9fe806bb10 2024-09-13T15:43:39.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.715+0000 7f9fe5d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9fe806c430 0x7f9fe812d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.715+0000 7f9fe6d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fe812d930 0x7f9fe812dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.715+0000 7f9fe5d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9fe806c430 0x7f9fe812d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37130/0 (socket says 172.21.15.70:37130) 2024-09-13T15:43:39.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.715+0000 7f9fe5d9b640 1 -- 172.21.15.70:0/3510675822 learned_addr learned my addr 172.21.15.70:0/3510675822 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:39.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.715+0000 7f9fe6d9d640 1 --2- 172.21.15.70:0/3510675822 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fe812d930 0x7f9fe812dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:54292/0 (socket says 172.21.15.70:54292) 2024-09-13T15:43:39.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.715+0000 7f9fe659c640 1 --2- 172.21.15.70:0/3510675822 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9fe806bb10 0x7f9fe806bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.715+0000 7f9fe6d9d640 1 -- 172.21.15.70:0/3510675822 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9fe806bb10 msgr2=0x7f9fe806bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:39.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.715+0000 7f9fe6d9d640 1 --2- 172.21.15.70:0/3510675822 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9fe806bb10 0x7f9fe806bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.715+0000 7f9fe6d9d640 1 -- 172.21.15.70:0/3510675822 >> v2:172.21.15.70:3300/0 conn(0x7f9fe806c430 msgr2=0x7f9fe812d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:39.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.715+0000 7f9fe6d9d640 1 --2- 172.21.15.70:0/3510675822 >> v2:172.21.15.70:3300/0 conn(0x7f9fe806c430 0x7f9fe812d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.715+0000 7f9fe6d9d640 1 -- 172.21.15.70:0/3510675822 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe806c810 con 0x7f9fe812d930 2024-09-13T15:43:39.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.716+0000 7f9fe6d9d640 1 --2- 172.21.15.70:0/3510675822 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fe812d930 0x7f9fe812dda0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009d90 tx=0x7f9fe002a490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baef660c8a6eb20d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:39.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.716+0000 7f9fe559a640 1 -- 172.21.15.70:0/3510675822 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0007740 con 0x7f9fe812d930 2024-09-13T15:43:39.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.716+0000 7f9fe559a640 1 -- 172.21.15.70:0/3510675822 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe002d050 con 0x7f9fe812d930 2024-09-13T15:43:39.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.716+0000 7f9fe559a640 1 -- 172.21.15.70:0/3510675822 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0007ac0 con 0x7f9fe812d930 2024-09-13T15:43:39.718 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:39.717+0000 7f1d6571d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T15:43:39.719 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:39.717+0000 7fd9b5c49040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T15:43:39.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.717+0000 7f9fece4f640 1 -- 172.21.15.70:0/3510675822 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fe812d930 msgr2=0x7f9fe812dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:39.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.717+0000 7f9fece4f640 1 --2- 172.21.15.70:0/3510675822 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fe812d930 0x7f9fe812dda0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009d90 tx=0x7f9fe002a490 comp rx=0 tx=0).stop 2024-09-13T15:43:39.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.717+0000 7f9fece4f640 1 -- 172.21.15.70:0/3510675822 shutdown_connections 2024-09-13T15:43:39.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.717+0000 7f9fece4f640 1 --2- 172.21.15.70:0/3510675822 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fe812d930 0x7f9fe812dda0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.717+0000 7f9fece4f640 1 --2- 172.21.15.70:0/3510675822 >> v2:172.21.15.70:3300/0 conn(0x7f9fe806c430 0x7f9fe812d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.717+0000 7f9fece4f640 1 --2- 172.21.15.70:0/3510675822 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9fe806bb10 0x7f9fe806bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.717+0000 7f9fece4f640 1 -- 172.21.15.70:0/3510675822 >> 172.21.15.70:0/3510675822 conn(0x7f9fe809fe20 msgr2=0x7f9fe80a0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:39.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.717+0000 7f9fece4f640 1 -- 172.21.15.70:0/3510675822 shutdown_connections 2024-09-13T15:43:39.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.717+0000 7f9fece4f640 1 -- 172.21.15.70:0/3510675822 wait complete. 2024-09-13T15:43:39.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.718+0000 7f9fece4f640 1 Processor -- start 2024-09-13T15:43:39.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.718+0000 7f9fece4f640 1 -- start start 2024-09-13T15:43:39.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.719+0000 7f9fece4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fe806bb10 0x7f9fe8132510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.719+0000 7f9fece4f640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9fe806c430 0x7f9fe8132a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.719+0000 7f9fece4f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fe812d930 0x7f9fe8132f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:39.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.719+0000 7f9fece4f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81336b0 con 0x7f9fe806bb10 2024-09-13T15:43:39.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.719+0000 7f9fece4f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8127d30 con 0x7f9fe812d930 2024-09-13T15:43:39.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.719+0000 7f9fece4f640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8132250 con 0x7f9fe806c430 2024-09-13T15:43:39.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.720+0000 7f9fe659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fe806bb10 0x7f9fe8132510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.720+0000 7f9fe6d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fe812d930 0x7f9fe8132f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.720+0000 7f9fe659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fe806bb10 0x7f9fe8132510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37138/0 (socket says 172.21.15.70:37138) 2024-09-13T15:43:39.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.720+0000 7f9fe659c640 1 -- 172.21.15.70:0/2251786714 learned_addr learned my addr 172.21.15.70:0/2251786714 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:39.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.720+0000 7f9fe6d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fe812d930 0x7f9fe8132f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:54306/0 (socket says 172.21.15.70:54306) 2024-09-13T15:43:39.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.720+0000 7f9fe5d9b640 1 --2- 172.21.15.70:0/2251786714 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9fe806c430 0x7f9fe8132a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:39.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.720+0000 7f9fe659c640 1 -- 172.21.15.70:0/2251786714 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9fe806c430 msgr2=0x7f9fe8132a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:39.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.720+0000 7f9fe659c640 1 --2- 172.21.15.70:0/2251786714 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9fe806c430 0x7f9fe8132a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.720+0000 7f9fe659c640 1 -- 172.21.15.70:0/2251786714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fe812d930 msgr2=0x7f9fe8132f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:39.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.720+0000 7f9fe659c640 1 --2- 172.21.15.70:0/2251786714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fe812d930 0x7f9fe8132f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.723 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.720+0000 7f9fe659c640 1 -- 172.21.15.70:0/2251786714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe0009a70 con 0x7f9fe806bb10 2024-09-13T15:43:39.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.720+0000 7f9fe659c640 1 --2- 172.21.15.70:0/2251786714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fe806bb10 0x7f9fe8132510 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800b170 tx=0x7f9fd800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:39.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.720+0000 7f9fd6ffd640 1 -- 172.21.15.70:0/2251786714 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8019070 con 0x7f9fe806bb10 2024-09-13T15:43:39.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.720+0000 7f9fd6ffd640 1 -- 172.21.15.70:0/2251786714 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd800be60 con 0x7f9fe806bb10 2024-09-13T15:43:39.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.721+0000 7f9fd6ffd640 1 -- 172.21.15.70:0/2251786714 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8014af0 con 0x7f9fe806bb10 2024-09-13T15:43:39.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.721+0000 7f9fece4f640 1 -- 172.21.15.70:0/2251786714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe81318c0 con 0x7f9fe806bb10 2024-09-13T15:43:39.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.721+0000 7f9fece4f640 1 -- 172.21.15.70:0/2251786714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe8130fd0 con 0x7f9fe806bb10 2024-09-13T15:43:39.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.722+0000 7f9fece4f640 1 -- 172.21.15.70:0/2251786714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe812d3f0 con 0x7f9fe806bb10 2024-09-13T15:43:39.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.722+0000 7f9fd6ffd640 1 -- 172.21.15.70:0/2251786714 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f9fd801b760 con 0x7f9fe806bb10 2024-09-13T15:43:39.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.723+0000 7f9fd6ffd640 1 -- 172.21.15.70:0/2251786714 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f9fd806b2a0 con 0x7f9fe806bb10 2024-09-13T15:43:39.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.723+0000 7f9fe5d9b640 1 -- 172.21.15.70:0/2251786714 >> v1:172.21.15.70:6816/32242 conn(0x7f9fc804cfc0 legacy=0x7f9fc804f480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6816/32242 2024-09-13T15:43:39.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.729+0000 7f9fd6ffd640 1 -- 172.21.15.70:0/2251786714 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f9fd8046100 con 0x7f9fe806bb10 2024-09-13T15:43:39.767 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:39.766+0000 7fd9b5c49040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T15:43:39.768 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:39.767+0000 7f1d6571d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T15:43:39.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.868+0000 7f9fece4f640 1 -- 172.21.15.70:0/2251786714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fe812a660 con 0x7f9fe806bb10 2024-09-13T15:43:39.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.869+0000 7f9fd6ffd640 1 -- 172.21.15.70:0/2251786714 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+8596 (secure 0 0 0) 0x7f9fd8022020 con 0x7f9fe806bb10 2024-09-13T15:43:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:43:39.870 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":18,"fsid":"45bbcdad-f4fb-4639-ab42-ccda8de151dd","created":"2024-09-13T15:42:09.863982+0000","modified":"2024-09-13T15:42:34.654949+0000","last_up_change":"2024-09-13T15:42:20.070768+0000","last_in_change":"2024-09-13T15:42:11.714506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T15:42:21.000499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T15:42:26.149909+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"265c6f1d-08a7-4c43-b2a6-2417ab814825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":32759}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":32759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":32759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":32759}]},"public_addr":"172.21.15.70:6804/32759","cluster_addr":"172.21.15.70:6805/32759","heartbeat_back_addr":"172.21.15.70:6807/32759","heartbeat_front_addr":"172.21.15.70:6806/32759","state":["exists","up"]},{"osd":1,"uuid":"ba4d90f9-6972-45df-800f-92ad808b06ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":32758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":32758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":32758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":32758}]},"public_addr":"172.21.15.70:6800/32758","cluster_addr":"172.21.15.70:6801/32758","heartbeat_back_addr":"172.21.15.70:6803/32758","heartbeat_front_addr":"172.21.15.70:6802/32758","state":["exists","up"]},{"osd":2,"uuid":"b74d8eb5-3082-4d69-a755-7f5f92dec5b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":32760}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":32760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":32760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":32760}]},"public_addr":"172.21.15.70:6812/32760","cluster_addr":"172.21.15.70:6813/32760","heartbeat_back_addr":"172.21.15.70:6815/32760","heartbeat_front_addr":"172.21.15.70:6814/32760","state":["exists","up"]},{"osd":3,"uuid":"02835b7d-e2cf-4ef8-87ba-b225593689d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":32769}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":32769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":32769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":32769}]},"public_addr":"172.21.15.70:6808/32769","cluster_addr":"172.21.15.70:6809/32769","heartbeat_back_addr":"172.21.15.70:6811/32769","heartbeat_front_addr":"172.21.15.70:6810/32769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.853420+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.807491+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.925567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.883852+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T15:43:39.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.871+0000 7f9fece4f640 1 -- 172.21.15.70:0/2251786714 >> v1:172.21.15.70:6816/32242 conn(0x7f9fc804cfc0 legacy=0x7f9fc804f480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:43:39.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.871+0000 7f9fece4f640 1 -- 172.21.15.70:0/2251786714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fe806bb10 msgr2=0x7f9fe8132510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:39.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.871+0000 7f9fece4f640 1 --2- 172.21.15.70:0/2251786714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fe806bb10 0x7f9fe8132510 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800b170 tx=0x7f9fd800b630 comp rx=0 tx=0).stop 2024-09-13T15:43:39.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.871+0000 7f9fece4f640 1 -- 172.21.15.70:0/2251786714 shutdown_connections 2024-09-13T15:43:39.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.871+0000 7f9fece4f640 1 --2- 172.21.15.70:0/2251786714 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9fe812d930 0x7f9fe8132f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.871+0000 7f9fece4f640 1 --2- 172.21.15.70:0/2251786714 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9fe806c430 0x7f9fe8132a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.871+0000 7f9fece4f640 1 --2- 172.21.15.70:0/2251786714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fe806bb10 0x7f9fe8132510 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:39.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.871+0000 7f9fece4f640 1 -- 172.21.15.70:0/2251786714 >> 172.21.15.70:0/2251786714 conn(0x7f9fe809fe20 msgr2=0x7f9fe8127330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:39.874 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.871+0000 7f9fece4f640 1 -- 172.21.15.70:0/2251786714 shutdown_connections 2024-09-13T15:43:39.874 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:39.871+0000 7f9fece4f640 1 -- 172.21.15.70:0/2251786714 wait complete. 2024-09-13T15:43:39.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T15:43:39.886 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T15:43:39.886 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T15:43:39.887 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T15:43:39.930 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:39.929+0000 7fd9b5c49040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T15:43:39.930 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:39.929+0000 7f1d6571d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T15:43:40.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.001+0000 7f1e701ce640 1 Processor -- start 2024-09-13T15:43:40.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.001+0000 7f1e701ce640 1 -- start start 2024-09-13T15:43:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.002+0000 7f1e701ce640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1e6806cea0 0x7f1e6806d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.002+0000 7f1e701ce640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e6806d7c0 0x7f1e6812db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.002+0000 7f1e701ce640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e68096df0 con 0x7f1e6812e080 2024-09-13T15:43:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.002+0000 7f1e701ce640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6806e390 con 0x7f1e6806cea0 2024-09-13T15:43:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.002+0000 7f1e701ce640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6806b4d0 con 0x7f1e6806d7c0 2024-09-13T15:43:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.002+0000 7f1e6d769640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e6806d7c0 0x7f1e6812db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.002+0000 7f1e6e76b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1e6812e080 0x7f1e68134d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58100/0 (socket says 172.21.15.70:58100) 2024-09-13T15:43:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.002+0000 7f1e6e76b640 1 -- 172.21.15.70:0/1047210355 learned_addr learned my addr 172.21.15.70:0/1047210355 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.002+0000 7f1e6df6a640 1 --2- 172.21.15.70:0/1047210355 >> v2:172.21.15.70:3300/0 conn(0x7f1e6806cea0 0x7f1e6806d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.003+0000 7f1e6cf68640 1 -- 172.21.15.70:0/1047210355 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48752263 0 0) 0x7f1e68096df0 con 0x7f1e6812e080 2024-09-13T15:43:40.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.003+0000 7f1e6cf68640 1 -- 172.21.15.70:0/1047210355 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e54003520 con 0x7f1e6812e080 2024-09-13T15:43:40.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.003+0000 7f1e6d769640 1 -- 172.21.15.70:0/1047210355 >> v2:172.21.15.70:3300/0 conn(0x7f1e6806cea0 msgr2=0x7f1e6806d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:40.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.003+0000 7f1e6d769640 1 --2- 172.21.15.70:0/1047210355 >> v2:172.21.15.70:3300/0 conn(0x7f1e6806cea0 0x7f1e6806d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.003+0000 7f1e6d769640 1 -- 172.21.15.70:0/1047210355 >> v1:172.21.15.70:6789/0 conn(0x7f1e6812e080 legacy=0x7f1e68134d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.003+0000 7f1e6d769640 1 -- 172.21.15.70:0/1047210355 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e68071ab0 con 0x7f1e6806d7c0 2024-09-13T15:43:40.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.004+0000 7f1e6d769640 1 --2- 172.21.15.70:0/1047210355 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e6806d7c0 0x7f1e6812db40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1e640097a0 tx=0x7f1e6402a010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71eb30e5c8039195 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.004+0000 7f1e6cf68640 1 -- 172.21.15.70:0/1047210355 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6402ad20 con 0x7f1e6806d7c0 2024-09-13T15:43:40.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.004+0000 7f1e6cf68640 1 -- 172.21.15.70:0/1047210355 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6402ae80 con 0x7f1e6806d7c0 2024-09-13T15:43:40.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.004+0000 7f1e6cf68640 1 -- 172.21.15.70:0/1047210355 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64032660 con 0x7f1e6806d7c0 2024-09-13T15:43:40.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.005+0000 7f1e701ce640 1 -- 172.21.15.70:0/1047210355 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e6806d7c0 msgr2=0x7f1e6812db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.005+0000 7f1e701ce640 1 --2- 172.21.15.70:0/1047210355 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e6806d7c0 0x7f1e6812db40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1e640097a0 tx=0x7f1e6402a010 comp rx=0 tx=0).stop 2024-09-13T15:43:40.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.005+0000 7f1e701ce640 1 -- 172.21.15.70:0/1047210355 shutdown_connections 2024-09-13T15:43:40.009 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.005+0000 7f1e701ce640 1 --2- 172.21.15.70:0/1047210355 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e6806d7c0 0x7f1e6812db40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.005+0000 7f1e701ce640 1 --2- 172.21.15.70:0/1047210355 >> v2:172.21.15.70:3300/0 conn(0x7f1e6806cea0 0x7f1e6806d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.005+0000 7f1e701ce640 1 -- 172.21.15.70:0/1047210355 >> 172.21.15.70:0/1047210355 conn(0x7f1e6809cde0 msgr2=0x7f1e6809d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.006+0000 7f1e701ce640 1 -- 172.21.15.70:0/1047210355 shutdown_connections 2024-09-13T15:43:40.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.006+0000 7f1e701ce640 1 -- 172.21.15.70:0/1047210355 wait complete. 2024-09-13T15:43:40.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.007+0000 7f1e701ce640 1 Processor -- start 2024-09-13T15:43:40.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.007+0000 7f1e701ce640 1 -- start start 2024-09-13T15:43:40.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.007+0000 7f1e701ce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e6806cea0 0x7f1e68122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.007+0000 7f1e701ce640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e6806d7c0 0x7f1e68122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.007+0000 7f1e701ce640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e6812e080 0x7f1e68123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.007+0000 7f1e701ce640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e68123ee0 con 0x7f1e6806cea0 2024-09-13T15:43:40.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.007+0000 7f1e701ce640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e680718a0 con 0x7f1e6812e080 2024-09-13T15:43:40.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.007+0000 7f1e701ce640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e68123740 con 0x7f1e6806d7c0 2024-09-13T15:43:40.011 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.007+0000 7f1e6df6a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e6806cea0 0x7f1e68122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.011 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.008+0000 7f1e6df6a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e6806cea0 0x7f1e68122780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37156/0 (socket says 172.21.15.70:37156) 2024-09-13T15:43:40.011 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.008+0000 7f1e6df6a640 1 -- 172.21.15.70:0/2924164697 learned_addr learned my addr 172.21.15.70:0/2924164697 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:40.011 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.008+0000 7f1e6d769640 1 --2- 172.21.15.70:0/2924164697 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e6806d7c0 0x7f1e68122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.011 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.008+0000 7f1e6e76b640 1 --2- 172.21.15.70:0/2924164697 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e6812e080 0x7f1e68123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.008+0000 7f1e6df6a640 1 -- 172.21.15.70:0/2924164697 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e6806d7c0 msgr2=0x7f1e68122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.008+0000 7f1e6df6a640 1 --2- 172.21.15.70:0/2924164697 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1e6806d7c0 0x7f1e68122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.008+0000 7f1e6df6a640 1 -- 172.21.15.70:0/2924164697 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e6812e080 msgr2=0x7f1e68123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.008+0000 7f1e6df6a640 1 --2- 172.21.15.70:0/2924164697 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1e6812e080 0x7f1e68123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.008+0000 7f1e6df6a640 1 -- 172.21.15.70:0/2924164697 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e64009400 con 0x7f1e6806cea0 2024-09-13T15:43:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.009+0000 7f1e6df6a640 1 --2- 172.21.15.70:0/2924164697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e6806cea0 0x7f1e68122780 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00e9b0 tx=0x7f1e5c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.009+0000 7f1e5a7fc640 1 -- 172.21.15.70:0/2924164697 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c009670 con 0x7f1e6806cea0 2024-09-13T15:43:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.009+0000 7f1e5a7fc640 1 -- 172.21.15.70:0/2924164697 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5c00c9a0 con 0x7f1e6806cea0 2024-09-13T15:43:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.009+0000 7f1e5a7fc640 1 -- 172.21.15.70:0/2924164697 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c00cd50 con 0x7f1e6806cea0 2024-09-13T15:43:40.014 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.009+0000 7f1e701ce640 1 -- 172.21.15.70:0/2924164697 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6806bc10 con 0x7f1e6806cea0 2024-09-13T15:43:40.014 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.009+0000 7f1e701ce640 1 -- 172.21.15.70:0/2924164697 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e681250f0 con 0x7f1e6806cea0 2024-09-13T15:43:40.014 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.010+0000 7f1e701ce640 1 -- 172.21.15.70:0/2924164697 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e681253a0 con 0x7f1e6806cea0 2024-09-13T15:43:40.014 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.011+0000 7f1e5a7fc640 1 -- 172.21.15.70:0/2924164697 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f1e5c013470 con 0x7f1e6806cea0 2024-09-13T15:43:40.014 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.012+0000 7f1e5a7fc640 1 -- 172.21.15.70:0/2924164697 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f1e5c0639f0 con 0x7f1e6806cea0 2024-09-13T15:43:40.014 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.012+0000 7f1e5a7fc640 1 -- 172.21.15.70:0/2924164697 --> v1:172.21.15.70:6812/32760 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e4c057350 con 0x7f1e4c053680 2024-09-13T15:43:40.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.012+0000 7f1e5a7fc640 1 -- 172.21.15.70:0/2924164697 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e5c063da0 con 0x7f1e6806cea0 2024-09-13T15:43:40.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.012+0000 7f1e6d769640 1 -- 172.21.15.70:0/2924164697 >> v1:172.21.15.70:6816/32242 conn(0x7f1e4c04cff0 legacy=0x7f1e4c04f4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6816/32242 2024-09-13T15:43:40.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.013+0000 7f1e5a7fc640 1 -- 172.21.15.70:0/2924164697 <== osd.2 v1:172.21.15.70:6812/32760 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f1e4c057350 con 0x7f1e4c053680 2024-09-13T15:43:40.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.037+0000 7f1e701ce640 1 -- 172.21.15.70:0/2924164697 --> v1:172.21.15.70:6812/32760 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e681253a0 con 0x7f1e4c053680 2024-09-13T15:43:40.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.037+0000 7f1e5a7fc640 1 -- 172.21.15.70:0/2924164697 <== osd.2 v1:172.21.15.70:6812/32760 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1159357919) 0x7f1e681253a0 con 0x7f1e4c053680 2024-09-13T15:43:40.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.038+0000 7f1e701ce640 1 -- 172.21.15.70:0/2924164697 >> v1:172.21.15.70:6812/32760 conn(0x7f1e4c053680 legacy=0x7f1e4c055ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.038+0000 7f1e701ce640 1 -- 172.21.15.70:0/2924164697 >> v1:172.21.15.70:6816/32242 conn(0x7f1e4c04cff0 legacy=0x7f1e4c04f4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:43:40.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.038+0000 7f1e701ce640 1 -- 172.21.15.70:0/2924164697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e6806cea0 msgr2=0x7f1e68122780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.038+0000 7f1e701ce640 1 --2- 172.21.15.70:0/2924164697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e6806cea0 0x7f1e68122780 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00e9b0 tx=0x7f1e5c00ee70 comp rx=0 tx=0).stop 2024-09-13T15:43:40.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.038+0000 7f1e6e76b640 1 -- 172.21.15.70:0/2924164697 reap_dead start 2024-09-13T15:43:40.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.038+0000 7f1e701ce640 1 -- 172.21.15.70:0/2924164697 shutdown_connections 2024-09-13T15:43:40.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.038+0000 7f1e701ce640 1 -- 172.21.15.70:0/2924164697 >> 172.21.15.70:0/2924164697 conn(0x7f1e6809cde0 msgr2=0x7f1e68070fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.038+0000 7f1e701ce640 1 -- 172.21.15.70:0/2924164697 shutdown_connections 2024-09-13T15:43:40.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.038+0000 7f1e701ce640 1 -- 172.21.15.70:0/2924164697 wait complete. 2024-09-13T15:43:40.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.041+0000 7fe859965640 1 Processor -- start 2024-09-13T15:43:40.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.041+0000 7fe859965640 1 -- start start 2024-09-13T15:43:40.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.042+0000 7fe859965640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe854090380 0x7fe854090760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.042+0000 7fe859965640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe854090ca0 0x7fe8541319b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.042+0000 7fe859965640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe854091080 0x7fe854131ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.042+0000 7fe859965640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe854066d20 con 0x7fe854090380 2024-09-13T15:43:40.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.042+0000 7fe859965640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe854091810 con 0x7fe854090ca0 2024-09-13T15:43:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.042+0000 7fe859965640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe854099aa0 con 0x7fe854091080 2024-09-13T15:43:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.042+0000 7fe852ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe854090380 0x7fe854090760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.042+0000 7fe8537fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe854091080 0x7fe854131ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.042+0000 7fe852ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe854090380 0x7fe854090760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37162/0 (socket says 172.21.15.70:37162) 2024-09-13T15:43:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.042+0000 7fe852ffd640 1 -- 172.21.15.70:0/4078039910 learned_addr learned my addr 172.21.15.70:0/4078039910 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.043+0000 7fe8527fc640 1 --2- 172.21.15.70:0/4078039910 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe854090ca0 0x7fe8541319b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.043+0000 7fe852ffd640 1 -- 172.21.15.70:0/4078039910 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe854091080 msgr2=0x7fe854131ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.043+0000 7fe852ffd640 1 --2- 172.21.15.70:0/4078039910 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe854091080 0x7fe854131ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.043+0000 7fe852ffd640 1 -- 172.21.15.70:0/4078039910 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe854090ca0 msgr2=0x7fe8541319b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:40.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.044+0000 7fe852ffd640 1 --2- 172.21.15.70:0/4078039910 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe854090ca0 0x7fe8541319b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.044+0000 7fe852ffd640 1 -- 172.21.15.70:0/4078039910 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe854065e40 con 0x7fe854090380 2024-09-13T15:43:40.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.044+0000 7fe8527fc640 1 --2- 172.21.15.70:0/4078039910 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe854090ca0 0x7fe8541319b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:43:40.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.045+0000 7fe852ffd640 1 --2- 172.21.15.70:0/4078039910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe854090380 0x7fe854090760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009e50 tx=0x7fe84c02a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93262d35122d92cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.045+0000 7fe851ffb640 1 -- 172.21.15.70:0/4078039910 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c031600 con 0x7fe854090380 2024-09-13T15:43:40.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.045+0000 7fe851ffb640 1 -- 172.21.15.70:0/4078039910 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c02d050 con 0x7fe854090380 2024-09-13T15:43:40.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.046+0000 7fe851ffb640 1 -- 172.21.15.70:0/4078039910 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c031980 con 0x7fe854090380 2024-09-13T15:43:40.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.046+0000 7fe859965640 1 -- 172.21.15.70:0/4078039910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe854090380 msgr2=0x7fe854090760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.046+0000 7fe859965640 1 --2- 172.21.15.70:0/4078039910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe854090380 0x7fe854090760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009e50 tx=0x7fe84c02a550 comp rx=0 tx=0).stop 2024-09-13T15:43:40.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.046+0000 7fe859965640 1 -- 172.21.15.70:0/4078039910 shutdown_connections 2024-09-13T15:43:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.046+0000 7fe859965640 1 --2- 172.21.15.70:0/4078039910 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe854091080 0x7fe854131ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.046+0000 7fe859965640 1 --2- 172.21.15.70:0/4078039910 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe854090ca0 0x7fe8541319b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.046+0000 7fe859965640 1 --2- 172.21.15.70:0/4078039910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe854090380 0x7fe854090760 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.046+0000 7fe859965640 1 -- 172.21.15.70:0/4078039910 >> 172.21.15.70:0/4078039910 conn(0x7fe85409a140 msgr2=0x7fe85409a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.047+0000 7fe859965640 1 -- 172.21.15.70:0/4078039910 shutdown_connections 2024-09-13T15:43:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.047+0000 7fe859965640 1 -- 172.21.15.70:0/4078039910 wait complete. 2024-09-13T15:43:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.048+0000 7fe859965640 1 Processor -- start 2024-09-13T15:43:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.048+0000 7fe859965640 1 -- start start 2024-09-13T15:43:40.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.049+0000 7fe859965640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe854090380 0x7fe8541351e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.049+0000 7fe859965640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe854090ca0 0x7fe854135720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.051 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.049+0000 7fe859965640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe854091080 0x7fe854135c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.049+0000 7fe859965640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe854136c40 con 0x7fe854091080 2024-09-13T15:43:40.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.049+0000 7fe859965640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85406c400 con 0x7fe854090380 2024-09-13T15:43:40.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.049+0000 7fe859965640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe85406c900 con 0x7fe854090ca0 2024-09-13T15:43:40.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.049+0000 7fe8527fc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe854090ca0 0x7fe854135720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.049+0000 7fe8527fc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe854090ca0 0x7fe854135720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:37874/0 (socket says 172.21.15.70:37874) 2024-09-13T15:43:40.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.049+0000 7fe8527fc640 1 -- 172.21.15.70:0/3532896870 learned_addr learned my addr 172.21.15.70:0/3532896870 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:40.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.049+0000 7fe852ffd640 1 --2- 172.21.15.70:0/3532896870 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe854090380 0x7fe8541351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.052 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.050+0000 7f36ab841640 1 Processor -- start 2024-09-13T15:43:40.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.050+0000 7fe852ffd640 1 -- 172.21.15.70:0/3532896870 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe854090ca0 msgr2=0x7fe854135720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.050+0000 7fe852ffd640 1 --2- 172.21.15.70:0/3532896870 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe854090ca0 0x7fe854135720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.050+0000 7fe852ffd640 1 -- 172.21.15.70:0/3532896870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe854091080 msgr2=0x7fe854135c60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:43:40.053 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.050+0000 7f36ab841640 1 -- start start 2024-09-13T15:43:40.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7fe8537fe640 1 --2- 172.21.15.70:0/3532896870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe854091080 0x7fe854135c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7fe852ffd640 1 --2- 172.21.15.70:0/3532896870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe854091080 0x7fe854135c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7feabfb08640 1 Processor -- start 2024-09-13T15:43:40.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7fe852ffd640 1 -- 172.21.15.70:0/3532896870 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c009b30 con 0x7fe854090380 2024-09-13T15:43:40.055 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7f36ab841640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f36a4070c30 0x7f36a4071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7fe8527fc640 1 --2- 172.21.15.70:0/3532896870 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe854090ca0 0x7fe854135720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7fe8537fe640 1 --2- 172.21.15.70:0/3532896870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe854091080 0x7fe854135c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:43:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7f36ab841640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f36a4071550 0x7f36a412b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7f36ab841640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36a412b6a0 0x7f36a412bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7f36a8ddc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f36a4071550 0x7f36a412b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7f36a8ddc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f36a4071550 0x7f36a412b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:37882/0 (socket says 172.21.15.70:37882) 2024-09-13T15:43:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7f36ab841640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a4070130 con 0x7f36a4070c30 2024-09-13T15:43:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7f36ab841640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a412c070 con 0x7f36a412b6a0 2024-09-13T15:43:40.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7f36ab841640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36a4070970 con 0x7f36a4071550 2024-09-13T15:43:40.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7f36a95dd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f36a4070c30 0x7f36a4071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7f36a8ddc640 1 -- 172.21.15.70:0/745797314 learned_addr learned my addr 172.21.15.70:0/745797314 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7feabfb08640 1 -- start start 2024-09-13T15:43:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.051+0000 7feabfb08640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feab8090e70 0x7feab8136e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.052+0000 7feabd8a4640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7feab8090380 0x7feab8090760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58104/0 (socket says 172.21.15.70:58104) 2024-09-13T15:43:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.052+0000 7feabfb08640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feab8137340 0x7feab8139730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.052+0000 7feabd0a3640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feab8090e70 0x7feab8136e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.052+0000 7feabd0a3640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feab8090e70 0x7feab8136e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:37888/0 (socket says 172.21.15.70:37888) 2024-09-13T15:43:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.052+0000 7feabd0a3640 1 -- 172.21.15.70:0/1423351805 learned_addr learned my addr 172.21.15.70:0/1423351805 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.052+0000 7f36a95dd640 1 -- 172.21.15.70:0/745797314 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f36a4071550 msgr2=0x7f36a412b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.052+0000 7f36a95dd640 1 --2- 172.21.15.70:0/745797314 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f36a4071550 0x7f36a412b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.052+0000 7f36a95dd640 1 -- 172.21.15.70:0/745797314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36a412b6a0 msgr2=0x7f36a412bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:40.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.052+0000 7f36a95dd640 1 --2- 172.21.15.70:0/745797314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36a412b6a0 0x7f36a412bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.052+0000 7f36a95dd640 1 -- 172.21.15.70:0/745797314 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a4071ce0 con 0x7f36a4070c30 2024-09-13T15:43:40.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.052+0000 7feabe0a5640 1 --2- 172.21.15.70:0/1423351805 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feab8137340 0x7feab8139730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.053+0000 7fe852ffd640 1 --2- 172.21.15.70:0/3532896870 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe854090380 0x7fe8541351e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe84c030000 tx=0x7fe84c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.052+0000 7feabfb08640 1 -- 172.21.15.70:0/1423351805 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab8091810 con 0x7feab8090380 2024-09-13T15:43:40.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.052+0000 7feabfb08640 1 -- 172.21.15.70:0/1423351805 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab8066d20 con 0x7feab8137340 2024-09-13T15:43:40.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.052+0000 7feabfb08640 1 -- 172.21.15.70:0/1423351805 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feab8099aa0 con 0x7feab8090e70 2024-09-13T15:43:40.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.052+0000 7feabc8a2640 1 -- 172.21.15.70:0/1423351805 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463221592 0 0) 0x7feab8091810 con 0x7feab8090380 2024-09-13T15:43:40.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.053+0000 7feabc8a2640 1 -- 172.21.15.70:0/1423351805 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaa4003520 con 0x7feab8090380 2024-09-13T15:43:40.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.053+0000 7f36a95dd640 1 --2- 172.21.15.70:0/745797314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a4070c30 0x7f36a4071010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f36940097a0 tx=0x7f3694029fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85557207444a1c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.053+0000 7fe83bfff640 1 -- 172.21.15.70:0/3532896870 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c03b610 con 0x7fe854090380 2024-09-13T15:43:40.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.053+0000 7fe859965640 1 -- 172.21.15.70:0/3532896870 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe854137390 con 0x7fe854090380 2024-09-13T15:43:40.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.053+0000 7feabd0a3640 1 -- 172.21.15.70:0/1423351805 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feab8137340 msgr2=0x7feab8139730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:40.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.053+0000 7fe859965640 1 -- 172.21.15.70:0/3532896870 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe854136380 con 0x7fe854090380 2024-09-13T15:43:40.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.053+0000 7feabd0a3640 1 --2- 172.21.15.70:0/1423351805 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feab8137340 0x7feab8139730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.053+0000 7feabd0a3640 1 -- 172.21.15.70:0/1423351805 >> v1:172.21.15.70:6789/0 conn(0x7feab8090380 legacy=0x7feab8090760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.053+0000 7feabd0a3640 1 -- 172.21.15.70:0/1423351805 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab8065e40 con 0x7feab8090e70 2024-09-13T15:43:40.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.053+0000 7fe83bfff640 1 -- 172.21.15.70:0/3532896870 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c03b770 con 0x7fe854090380 2024-09-13T15:43:40.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.053+0000 7fe83bfff640 1 -- 172.21.15.70:0/3532896870 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c03b610 con 0x7fe854090380 2024-09-13T15:43:40.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.054+0000 7f369bfff640 1 -- 172.21.15.70:0/745797314 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369402ad20 con 0x7f36a4070c30 2024-09-13T15:43:40.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.054+0000 7f369bfff640 1 -- 172.21.15.70:0/745797314 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369402ae80 con 0x7f36a4070c30 2024-09-13T15:43:40.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.054+0000 7f369bfff640 1 -- 172.21.15.70:0/745797314 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694032660 con 0x7f36a4070c30 2024-09-13T15:43:40.066 INFO:teuthology.orchestra.run.smithi070.stdout:38654705682 2024-09-13T15:43:40.066 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T15:43:40.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.055+0000 7fe859965640 1 -- 172.21.15.70:0/3532896870 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe834000fc0 con 0x7fe854090380 2024-09-13T15:43:40.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.055+0000 7f36ab841640 1 -- 172.21.15.70:0/745797314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a4070c30 msgr2=0x7f36a4071010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.055+0000 7f36ab841640 1 --2- 172.21.15.70:0/745797314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a4070c30 0x7f36a4071010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f36940097a0 tx=0x7f3694029fd0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.055+0000 7f36ab841640 1 -- 172.21.15.70:0/745797314 shutdown_connections 2024-09-13T15:43:40.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.055+0000 7f36ab841640 1 --2- 172.21.15.70:0/745797314 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36a412b6a0 0x7f36a412bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.055+0000 7f36ab841640 1 --2- 172.21.15.70:0/745797314 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f36a4071550 0x7f36a412b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.055+0000 7f36ab841640 1 --2- 172.21.15.70:0/745797314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a4070c30 0x7f36a4071010 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.055+0000 7f36ab841640 1 -- 172.21.15.70:0/745797314 >> 172.21.15.70:0/745797314 conn(0x7f36a409bc00 msgr2=0x7f36a409c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.056+0000 7f36ab841640 1 -- 172.21.15.70:0/745797314 shutdown_connections 2024-09-13T15:43:40.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.056+0000 7f36ab841640 1 -- 172.21.15.70:0/745797314 wait complete. 2024-09-13T15:43:40.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.056+0000 7feabd0a3640 1 --2- 172.21.15.70:0/1423351805 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feab8090e70 0x7feab8136e00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feab40098a0 tx=0x7feab402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8daaa7db4763b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.056+0000 7fe83bfff640 1 -- 172.21.15.70:0/3532896870 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7fe84c038050 con 0x7fe854090380 2024-09-13T15:43:40.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.056+0000 7feabc8a2640 1 -- 172.21.15.70:0/1423351805 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab402ae80 con 0x7feab8090e70 2024-09-13T15:43:40.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.056+0000 7feabc8a2640 1 -- 172.21.15.70:0/1423351805 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab4032440 con 0x7feab8090e70 2024-09-13T15:43:40.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.056+0000 7feabc8a2640 1 -- 172.21.15.70:0/1423351805 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40327b0 con 0x7feab8090e70 2024-09-13T15:43:40.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.057+0000 7fe8527fc640 1 -- 172.21.15.70:0/3532896870 >> v1:172.21.15.70:6816/32242 conn(0x7fe84004d0b0 legacy=0x7fe84004f570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6816/32242 2024-09-13T15:43:40.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.057+0000 7feabfb08640 1 -- 172.21.15.70:0/1423351805 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feab8090e70 msgr2=0x7feab8136e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.057+0000 7feabfb08640 1 --2- 172.21.15.70:0/1423351805 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feab8090e70 0x7feab8136e00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feab40098a0 tx=0x7feab402a130 comp rx=0 tx=0).stop 2024-09-13T15:43:40.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.057+0000 7fe83bfff640 1 -- 172.21.15.70:0/3532896870 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2836+0+0 (secure 0 0 0) 0x7fe84c083ce0 con 0x7fe854090380 2024-09-13T15:43:40.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.057+0000 7feabfb08640 1 -- 172.21.15.70:0/1423351805 shutdown_connections 2024-09-13T15:43:40.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.057+0000 7feabfb08640 1 --2- 172.21.15.70:0/1423351805 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feab8137340 0x7feab8139730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.057+0000 7feabfb08640 1 --2- 172.21.15.70:0/1423351805 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feab8090e70 0x7feab8136e00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.057+0000 7feabfb08640 1 -- 172.21.15.70:0/1423351805 >> 172.21.15.70:0/1423351805 conn(0x7feab809a140 msgr2=0x7feab809a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.057+0000 7feabfb08640 1 -- 172.21.15.70:0/1423351805 shutdown_connections 2024-09-13T15:43:40.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.057+0000 7feabfb08640 1 -- 172.21.15.70:0/1423351805 wait complete. 2024-09-13T15:43:40.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.057+0000 7f36ab841640 1 Processor -- start 2024-09-13T15:43:40.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.057+0000 7fe83bfff640 1 -- 172.21.15.70:0/3532896870 --> v1:172.21.15.70:6800/32758 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe840057410 con 0x7fe840053740 2024-09-13T15:43:40.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.057+0000 7fe83bfff640 1 -- 172.21.15.70:0/3532896870 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe84c03b610 con 0x7fe854090380 2024-09-13T15:43:40.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.058+0000 7f36ab841640 1 -- start start 2024-09-13T15:43:40.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.058+0000 7f36ab841640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f36a4070c30 0x7f36a412c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.058+0000 7f36ab841640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a4071550 0x7f36a412c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.058+0000 7f36a95dd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f36a4070c30 0x7f36a412c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.058+0000 7feabfb08640 1 Processor -- start 2024-09-13T15:43:40.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.058+0000 7f36a95dd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f36a4070c30 0x7f36a412c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:37902/0 (socket says 172.21.15.70:37902) 2024-09-13T15:43:40.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.058+0000 7f36a8ddc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a4071550 0x7f36a412c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7feabfb08640 1 -- start start 2024-09-13T15:43:40.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7fe83bfff640 1 -- 172.21.15.70:0/3532896870 <== osd.1 v1:172.21.15.70:6800/32758 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fe840057410 con 0x7fe840053740 2024-09-13T15:43:40.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7f36a8ddc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a4071550 0x7f36a412c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37184/0 (socket says 172.21.15.70:37184) 2024-09-13T15:43:40.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7f36ab841640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36a412b6a0 0x7f36a412cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7f36a8ddc640 1 -- 172.21.15.70:0/2632706984 learned_addr learned my addr 172.21.15.70:0/2632706984 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:40.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7f36ab841640 1 -- 172.21.15.70:0/2632706984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a412e480 con 0x7f36a4071550 2024-09-13T15:43:40.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7f36ab841640 1 -- 172.21.15.70:0/2632706984 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a412dca0 con 0x7f36a412b6a0 2024-09-13T15:43:40.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7f36ab841640 1 -- 172.21.15.70:0/2632706984 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36a412d4a0 con 0x7f36a4070c30 2024-09-13T15:43:40.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7f36a9dde640 1 --2- 172.21.15.70:0/2632706984 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36a412b6a0 0x7f36a412cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7f36a8ddc640 1 -- 172.21.15.70:0/2632706984 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f36a4070c30 msgr2=0x7f36a412c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7f36a8ddc640 1 --2- 172.21.15.70:0/2632706984 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f36a4070c30 0x7f36a412c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7f36a8ddc640 1 -- 172.21.15.70:0/2632706984 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36a412b6a0 msgr2=0x7f36a412cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7f36a8ddc640 1 --2- 172.21.15.70:0/2632706984 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f36a412b6a0 0x7f36a412cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7f36a8ddc640 1 -- 172.21.15.70:0/2632706984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3694009400 con 0x7f36a4071550 2024-09-13T15:43:40.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7feabfb08640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feab8090380 0x7feab8129f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7feabfb08640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feab8090e70 0x7feab812a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7feabfb08640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feab8137340 0x7feab812a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7feabfb08640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab8136e50 con 0x7feab8090380 2024-09-13T15:43:40.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7feabfb08640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab812ca70 con 0x7feab8090e70 2024-09-13T15:43:40.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7feabfb08640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feab8135ea0 con 0x7feab8137340 2024-09-13T15:43:40.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7feabd8a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feab8090380 0x7feab8129f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7feabd8a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feab8090380 0x7feab8129f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37194/0 (socket says 172.21.15.70:37194) 2024-09-13T15:43:40.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7feabd8a4640 1 -- 172.21.15.70:0/3560214117 learned_addr learned my addr 172.21.15.70:0/3560214117 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:40.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7feabe0a5640 1 --2- 172.21.15.70:0/3560214117 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feab8137340 0x7feab812a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7f36a8ddc640 1 --2- 172.21.15.70:0/2632706984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a4071550 0x7f36a412c720 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f36a000e9a0 tx=0x7f36a000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.079 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.059+0000 7feabd0a3640 1 --2- 172.21.15.70:0/3560214117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feab8090e70 0x7feab812a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.079 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.060+0000 7f3699ffb640 1 -- 172.21.15.70:0/2632706984 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a000c930 con 0x7f36a4071550 2024-09-13T15:43:40.079 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.060+0000 7feabe0a5640 1 -- 172.21.15.70:0/3560214117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feab8090e70 msgr2=0x7feab812a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.060+0000 7feabe0a5640 1 --2- 172.21.15.70:0/3560214117 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feab8090e70 0x7feab812a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.060+0000 7feabe0a5640 1 -- 172.21.15.70:0/3560214117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feab8090380 msgr2=0x7feab8129f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.060+0000 7feabe0a5640 1 --2- 172.21.15.70:0/3560214117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feab8090380 0x7feab8129f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.060+0000 7feabe0a5640 1 -- 172.21.15.70:0/3560214117 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab40094d0 con 0x7feab8137340 2024-09-13T15:43:40.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.060+0000 7f3699ffb640 1 -- 172.21.15.70:0/2632706984 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a000ca90 con 0x7f36a4071550 2024-09-13T15:43:40.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.060+0000 7f3699ffb640 1 -- 172.21.15.70:0/2632706984 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a000ce00 con 0x7f36a4071550 2024-09-13T15:43:40.080 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.060+0000 7f36a95dd640 1 --2- 172.21.15.70:0/2632706984 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f36a4070c30 0x7f36a412c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:43:40.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.060+0000 7f36ab841640 1 -- 172.21.15.70:0/2632706984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a412e850 con 0x7f36a4071550 2024-09-13T15:43:40.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.060+0000 7feabe0a5640 1 --2- 172.21.15.70:0/3560214117 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feab8137340 0x7feab812a980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feab000ef90 tx=0x7feab000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.061+0000 7feaaa7fc640 1 -- 172.21.15.70:0/3560214117 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab0009b10 con 0x7feab8137340 2024-09-13T15:43:40.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.061+0000 7feaaa7fc640 1 -- 172.21.15.70:0/3560214117 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab000f040 con 0x7feab8137340 2024-09-13T15:43:40.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.061+0000 7feaaa7fc640 1 -- 172.21.15.70:0/3560214117 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab00146a0 con 0x7feab8137340 2024-09-13T15:43:40.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.061+0000 7feabfb08640 1 -- 172.21.15.70:0/3560214117 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab812d250 con 0x7feab8137340 2024-09-13T15:43:40.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.061+0000 7feabfb08640 1 -- 172.21.15.70:0/3560214117 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab812b920 con 0x7feab8137340 2024-09-13T15:43:40.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.061+0000 7f36ab841640 1 -- 172.21.15.70:0/2632706984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a4127d40 con 0x7f36a4071550 2024-09-13T15:43:40.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.062+0000 7feabfb08640 1 -- 172.21.15.70:0/3560214117 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feab8091250 con 0x7feab8137340 2024-09-13T15:43:40.082 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.063+0000 7f3699ffb640 1 -- 172.21.15.70:0/2632706984 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f36a0013410 con 0x7f36a4071550 2024-09-13T15:43:40.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.063+0000 7feaaa7fc640 1 -- 172.21.15.70:0/3560214117 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7feab0014800 con 0x7feab8137340 2024-09-13T15:43:40.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.063+0000 7f36ab841640 1 -- 172.21.15.70:0/2632706984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f36a4132490 con 0x7f36a4071550 2024-09-13T15:43:40.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.063+0000 7feabd8a4640 1 -- 172.21.15.70:0/3560214117 >> v1:172.21.15.70:6816/32242 conn(0x7fea9c04cff0 legacy=0x7fea9c04f4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6816/32242 2024-09-13T15:43:40.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.063+0000 7feaaa7fc640 1 -- 172.21.15.70:0/3560214117 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2836+0+0 (secure 0 0 0) 0x7feab00640a0 con 0x7feab8137340 2024-09-13T15:43:40.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.063+0000 7f36a95dd640 1 -- 172.21.15.70:0/2632706984 >> v1:172.21.15.70:6816/32242 conn(0x7f368804d010 legacy=0x7f368804f4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6816/32242 2024-09-13T15:43:40.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.063+0000 7feaaa7fc640 1 -- 172.21.15.70:0/3560214117 --> v1:172.21.15.70:6804/32759 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea9c057350 con 0x7fea9c053680 2024-09-13T15:43:40.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.063+0000 7feaaa7fc640 1 -- 172.21.15.70:0/3560214117 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7feab003ee60 con 0x7feab8137340 2024-09-13T15:43:40.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.063+0000 7f3699ffb640 1 -- 172.21.15.70:0/2632706984 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f36a0066da0 con 0x7f36a4071550 2024-09-13T15:43:40.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.063+0000 7f3699ffb640 1 -- 172.21.15.70:0/2632706984 --> v1:172.21.15.70:6808/32769 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3688057370 con 0x7f36880536a0 2024-09-13T15:43:40.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.063+0000 7f3699ffb640 1 -- 172.21.15.70:0/2632706984 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36a006c4a0 con 0x7f36a4071550 2024-09-13T15:43:40.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.064+0000 7f3699ffb640 1 -- 172.21.15.70:0/2632706984 <== osd.3 v1:172.21.15.70:6808/32769 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f3688057370 con 0x7f36880536a0 2024-09-13T15:43:40.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.065+0000 7feaaa7fc640 1 -- 172.21.15.70:0/3560214117 <== osd.0 v1:172.21.15.70:6804/32759 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fea9c057350 con 0x7fea9c053680 2024-09-13T15:43:40.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.085+0000 7fe859965640 1 -- 172.21.15.70:0/3532896870 --> v1:172.21.15.70:6800/32758 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe834002bf0 con 0x7fe840053740 2024-09-13T15:43:40.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.086+0000 7fe83bfff640 1 -- 172.21.15.70:0/3532896870 <== osd.1 v1:172.21.15.70:6800/32758 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1447728683) 0x7fe834002bf0 con 0x7fe840053740 2024-09-13T15:43:40.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.086+0000 7fe859965640 1 -- 172.21.15.70:0/3532896870 >> v1:172.21.15.70:6800/32758 conn(0x7fe840053740 legacy=0x7fe840055ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.086+0000 7fe859965640 1 -- 172.21.15.70:0/3532896870 >> v1:172.21.15.70:6816/32242 conn(0x7fe84004d0b0 legacy=0x7fe84004f570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:43:40.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.086+0000 7fe859965640 1 -- 172.21.15.70:0/3532896870 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe854090380 msgr2=0x7fe8541351e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.086+0000 7fe859965640 1 --2- 172.21.15.70:0/3532896870 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe854090380 0x7fe8541351e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe84c030000 tx=0x7fe84c004a40 comp rx=0 tx=0).stop 2024-09-13T15:43:40.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.086+0000 7fe8537fe640 1 -- 172.21.15.70:0/3532896870 reap_dead start 2024-09-13T15:43:40.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.087+0000 7fe859965640 1 -- 172.21.15.70:0/3532896870 shutdown_connections 2024-09-13T15:43:40.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.087+0000 7fe859965640 1 -- 172.21.15.70:0/3532896870 >> 172.21.15.70:0/3532896870 conn(0x7fe85409a140 msgr2=0x7fe854096f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.087+0000 7fe859965640 1 -- 172.21.15.70:0/3532896870 shutdown_connections 2024-09-13T15:43:40.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.087+0000 7fe859965640 1 -- 172.21.15.70:0/3532896870 wait complete. 2024-09-13T15:43:40.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.089+0000 7feabfb08640 1 -- 172.21.15.70:0/3560214117 --> v1:172.21.15.70:6804/32759 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7feab8090bd0 con 0x7fea9c053680 2024-09-13T15:43:40.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.089+0000 7feaaa7fc640 1 -- 172.21.15.70:0/3560214117 <== osd.0 v1:172.21.15.70:6804/32759 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1159357919) 0x7feab8090bd0 con 0x7fea9c053680 2024-09-13T15:43:40.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.089+0000 7feabfb08640 1 -- 172.21.15.70:0/3560214117 >> v1:172.21.15.70:6804/32759 conn(0x7fea9c053680 legacy=0x7fea9c055ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.090+0000 7feabfb08640 1 -- 172.21.15.70:0/3560214117 >> v1:172.21.15.70:6816/32242 conn(0x7fea9c04cff0 legacy=0x7fea9c04f4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:43:40.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.090+0000 7feabfb08640 1 -- 172.21.15.70:0/3560214117 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feab8137340 msgr2=0x7feab812a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.090+0000 7feabfb08640 1 --2- 172.21.15.70:0/3560214117 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feab8137340 0x7feab812a980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7feab000ef90 tx=0x7feab000c490 comp rx=0 tx=0).stop 2024-09-13T15:43:40.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.090+0000 7feabe0a5640 1 -- 172.21.15.70:0/3560214117 reap_dead start 2024-09-13T15:43:40.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.090+0000 7feabfb08640 1 -- 172.21.15.70:0/3560214117 shutdown_connections 2024-09-13T15:43:40.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.090+0000 7feabfb08640 1 -- 172.21.15.70:0/3560214117 >> 172.21.15.70:0/3560214117 conn(0x7feab809a140 msgr2=0x7feab8096d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.090+0000 7feabfb08640 1 -- 172.21.15.70:0/3560214117 shutdown_connections 2024-09-13T15:43:40.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.090+0000 7feabfb08640 1 -- 172.21.15.70:0/3560214117 wait complete. 2024-09-13T15:43:40.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.094+0000 7f36ab841640 1 -- 172.21.15.70:0/2632706984 --> v1:172.21.15.70:6808/32769 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f36a412b160 con 0x7f36880536a0 2024-09-13T15:43:40.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.094+0000 7f3699ffb640 1 -- 172.21.15.70:0/2632706984 <== osd.3 v1:172.21.15.70:6808/32769 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1159357919) 0x7f36a412b160 con 0x7f36880536a0 2024-09-13T15:43:40.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.095+0000 7f36877fe640 1 -- 172.21.15.70:0/2632706984 >> v1:172.21.15.70:6808/32769 conn(0x7f36880536a0 legacy=0x7f3688055b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.096+0000 7f36877fe640 1 -- 172.21.15.70:0/2632706984 >> v1:172.21.15.70:6816/32242 conn(0x7f368804d010 legacy=0x7f368804f4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:43:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.096+0000 7f36877fe640 1 -- 172.21.15.70:0/2632706984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a4071550 msgr2=0x7f36a412c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.096+0000 7f36877fe640 1 --2- 172.21.15.70:0/2632706984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36a4071550 0x7f36a412c720 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f36a000e9a0 tx=0x7f36a000ee60 comp rx=0 tx=0).stop 2024-09-13T15:43:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.096+0000 7f36a9dde640 1 -- 172.21.15.70:0/2632706984 reap_dead start 2024-09-13T15:43:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.096+0000 7f36877fe640 1 -- 172.21.15.70:0/2632706984 shutdown_connections 2024-09-13T15:43:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.096+0000 7f36877fe640 1 -- 172.21.15.70:0/2632706984 >> 172.21.15.70:0/2632706984 conn(0x7f36a409bc00 msgr2=0x7f36a4129be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.096+0000 7f36877fe640 1 -- 172.21.15.70:0/2632706984 shutdown_connections 2024-09-13T15:43:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.096+0000 7f36877fe640 1 -- 172.21.15.70:0/2632706984 wait complete. 2024-09-13T15:43:40.109 INFO:teuthology.orchestra.run.smithi070.stdout:38654705681 2024-09-13T15:43:40.109 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T15:43:40.110 INFO:teuthology.orchestra.run.smithi070.stdout:38654705682 2024-09-13T15:43:40.110 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T15:43:40.116 INFO:teuthology.orchestra.run.smithi070.stdout:38654705682 2024-09-13T15:43:40.117 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T15:43:40.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.216+0000 7fb06aa5a640 1 Processor -- start 2024-09-13T15:43:40.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.216+0000 7fb06aa5a640 1 -- start start 2024-09-13T15:43:40.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.217+0000 7fb06aa5a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb06406c0f0 0x7fb06406c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.217+0000 7fb06aa5a640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb06412d960 0x7fb06412ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.217+0000 7fb06aa5a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb064073d10 con 0x7fb06406ca10 2024-09-13T15:43:40.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.217+0000 7fb06aa5a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06404e330 con 0x7fb06406c0f0 2024-09-13T15:43:40.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.217+0000 7fb06aa5a640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb06406ce30 con 0x7fb06412d960 2024-09-13T15:43:40.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.218+0000 7fb0637fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb06406ca10 0x7fb06412d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58114/0 (socket says 172.21.15.70:58114) 2024-09-13T15:43:40.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.218+0000 7fb0637fe640 1 -- 172.21.15.70:0/2907986850 learned_addr learned my addr 172.21.15.70:0/2907986850 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:40.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.218+0000 7fb068ff7640 1 --2- 172.21.15.70:0/2907986850 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb06412d960 0x7fb06412ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.218+0000 7fb062ffd640 1 -- 172.21.15.70:0/2907986850 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351498451 0 0) 0x7fb064073d10 con 0x7fb06406ca10 2024-09-13T15:43:40.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.218+0000 7fb062ffd640 1 -- 172.21.15.70:0/2907986850 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb04c003520 con 0x7fb06406ca10 2024-09-13T15:43:40.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.218+0000 7fb063fff640 1 --2- 172.21.15.70:0/2907986850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb06406c0f0 0x7fb06406c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.218+0000 7fb068ff7640 1 -- 172.21.15.70:0/2907986850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb06406c0f0 msgr2=0x7fb06406c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:40.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.218+0000 7fb068ff7640 1 --2- 172.21.15.70:0/2907986850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb06406c0f0 0x7fb06406c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.218+0000 7fb068ff7640 1 -- 172.21.15.70:0/2907986850 >> v1:172.21.15.70:6789/0 conn(0x7fb06406ca10 legacy=0x7fb06412d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.218+0000 7fb068ff7640 1 -- 172.21.15.70:0/2907986850 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0640978d0 con 0x7fb06412d960 2024-09-13T15:43:40.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.219+0000 7fb068ff7640 1 --2- 172.21.15.70:0/2907986850 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb06412d960 0x7fb06412ddd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00b080 tx=0x7fb05c02a780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eff6077742f41752 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.220+0000 7fb062ffd640 1 -- 172.21.15.70:0/2907986850 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c032640 con 0x7fb06412d960 2024-09-13T15:43:40.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.220+0000 7fb062ffd640 1 -- 172.21.15.70:0/2907986850 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05c0327a0 con 0x7fb06412d960 2024-09-13T15:43:40.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.220+0000 7fb062ffd640 1 -- 172.21.15.70:0/2907986850 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c032b30 con 0x7fb06412d960 2024-09-13T15:43:40.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.221+0000 7fb06aa5a640 1 -- 172.21.15.70:0/2907986850 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb06412d960 msgr2=0x7fb06412ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.221+0000 7fb06aa5a640 1 --2- 172.21.15.70:0/2907986850 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb06412d960 0x7fb06412ddd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00b080 tx=0x7fb05c02a780 comp rx=0 tx=0).stop 2024-09-13T15:43:40.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.221+0000 7fb06aa5a640 1 -- 172.21.15.70:0/2907986850 shutdown_connections 2024-09-13T15:43:40.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.221+0000 7fb06aa5a640 1 --2- 172.21.15.70:0/2907986850 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb06412d960 0x7fb06412ddd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.221+0000 7fb06aa5a640 1 --2- 172.21.15.70:0/2907986850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb06406c0f0 0x7fb06406c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.221+0000 7fb06aa5a640 1 -- 172.21.15.70:0/2907986850 >> 172.21.15.70:0/2907986850 conn(0x7fb0640a00d0 msgr2=0x7fb0640a04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.221+0000 7fb06aa5a640 1 -- 172.21.15.70:0/2907986850 shutdown_connections 2024-09-13T15:43:40.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.221+0000 7fb06aa5a640 1 -- 172.21.15.70:0/2907986850 wait complete. 2024-09-13T15:43:40.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.222+0000 7fb06aa5a640 1 Processor -- start 2024-09-13T15:43:40.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.223+0000 7fb06aa5a640 1 -- start start 2024-09-13T15:43:40.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.223+0000 7fb06aa5a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb06406c0f0 0x7fb064126fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.223+0000 7fb06aa5a640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb06406ca10 0x7fb064127510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.223+0000 7fb06aa5a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb06412d960 0x7fb0641c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.223+0000 7fb06aa5a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb064129cf0 con 0x7fb06406c0f0 2024-09-13T15:43:40.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.223+0000 7fb06aa5a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0641280c0 con 0x7fb06412d960 2024-09-13T15:43:40.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.223+0000 7fb06aa5a640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb06412a4c0 con 0x7fb06406ca10 2024-09-13T15:43:40.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.224+0000 7fb0637fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb06406ca10 0x7fb064127510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.224+0000 7fb0637fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb06406ca10 0x7fb064127510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:37928/0 (socket says 172.21.15.70:37928) 2024-09-13T15:43:40.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.224+0000 7fb0637fe640 1 -- 172.21.15.70:0/3004522749 learned_addr learned my addr 172.21.15.70:0/3004522749 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:40.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.224+0000 7fb0637fe640 1 -- 172.21.15.70:0/3004522749 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb06412d960 msgr2=0x7fb0641c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.224+0000 7fb068ff7640 1 --2- 172.21.15.70:0/3004522749 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb06412d960 0x7fb0641c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.224+0000 7fb063fff640 1 --2- 172.21.15.70:0/3004522749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb06406c0f0 0x7fb064126fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.224+0000 7fb0637fe640 1 --2- 172.21.15.70:0/3004522749 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb06412d960 0x7fb0641c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.224+0000 7fb0637fe640 1 -- 172.21.15.70:0/3004522749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb06406c0f0 msgr2=0x7fb064126fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.224+0000 7fb0637fe640 1 --2- 172.21.15.70:0/3004522749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb06406c0f0 0x7fb064126fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.224+0000 7fb0637fe640 1 -- 172.21.15.70:0/3004522749 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c009a70 con 0x7fb06406ca10 2024-09-13T15:43:40.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.225+0000 7fb068ff7640 1 --2- 172.21.15.70:0/3004522749 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb06412d960 0x7fb0641c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:43:40.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.225+0000 7fb063fff640 1 --2- 172.21.15.70:0/3004522749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb06406c0f0 0x7fb064126fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:43:40.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.225+0000 7fb0637fe640 1 --2- 172.21.15.70:0/3004522749 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb06406ca10 0x7fb064127510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb05800c530 tx=0x7fb05800c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.225+0000 7fb060ff9640 1 -- 172.21.15.70:0/3004522749 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058005ce0 con 0x7fb06406ca10 2024-09-13T15:43:40.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.225+0000 7fb060ff9640 1 -- 172.21.15.70:0/3004522749 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb058005e40 con 0x7fb06406ca10 2024-09-13T15:43:40.228 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.225+0000 7fb06aa5a640 1 -- 172.21.15.70:0/3004522749 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06412a780 con 0x7fb06406ca10 2024-09-13T15:43:40.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.225+0000 7fb060ff9640 1 -- 172.21.15.70:0/3004522749 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0580072b0 con 0x7fb06406ca10 2024-09-13T15:43:40.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.227+0000 7fb06aa5a640 1 -- 172.21.15.70:0/3004522749 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0641c6710 con 0x7fb06406ca10 2024-09-13T15:43:40.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.228+0000 7fb06aa5a640 1 -- 172.21.15.70:0/3004522749 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb06406c4d0 con 0x7fb06406ca10 2024-09-13T15:43:40.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.228+0000 7fb060ff9640 1 -- 172.21.15.70:0/3004522749 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7fb058007410 con 0x7fb06406ca10 2024-09-13T15:43:40.230 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.229+0000 7fb063fff640 1 -- 172.21.15.70:0/3004522749 >> v1:172.21.15.70:6816/32242 conn(0x7fb05004cfc0 legacy=0x7fb05004f480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6816/32242 2024-09-13T15:43:40.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.230+0000 7fb060ff9640 1 -- 172.21.15.70:0/3004522749 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2836+0+0 (secure 0 0 0) 0x7fb0580629f0 con 0x7fb06406ca10 2024-09-13T15:43:40.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.236+0000 7fb060ff9640 1 -- 172.21.15.70:0/3004522749 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7fb05803d870 con 0x7fb06406ca10 2024-09-13T15:43:40.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.310+0000 7f29e83bc640 1 Processor -- start 2024-09-13T15:43:40.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.310+0000 7f29e83bc640 1 -- start start 2024-09-13T15:43:40.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.311+0000 7f29e83bc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e006cb60 0x7f29e006cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.311+0000 7f29e83bc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29e006bb20 0x7f29e006d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.311+0000 7f29e83bc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e0071920 con 0x7f29e012dad0 2024-09-13T15:43:40.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.311+0000 7f29e83bc640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e006d9c0 con 0x7f29e006cb60 2024-09-13T15:43:40.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.311+0000 7f29e83bc640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0096ef0 con 0x7f29e006bb20 2024-09-13T15:43:40.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.312+0000 7f29e6158640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e006cb60 0x7f29e006cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.312+0000 7f29e6959640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f29e012dad0 0x7f29e012df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58120/0 (socket says 172.21.15.70:58120) 2024-09-13T15:43:40.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.312+0000 7f29e6959640 1 -- 172.21.15.70:0/366930360 learned_addr learned my addr 172.21.15.70:0/366930360 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:40.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.312+0000 7f29e5156640 1 -- 172.21.15.70:0/366930360 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823920278 0 0) 0x7f29e0071920 con 0x7f29e012dad0 2024-09-13T15:43:40.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.312+0000 7f29e5156640 1 -- 172.21.15.70:0/366930360 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29cc003520 con 0x7f29e012dad0 2024-09-13T15:43:40.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.312+0000 7f29e5957640 1 --2- 172.21.15.70:0/366930360 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29e006bb20 0x7f29e006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.312+0000 7f29e6158640 1 -- 172.21.15.70:0/366930360 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29e006bb20 msgr2=0x7f29e006d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:40.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.312+0000 7f29e6158640 1 --2- 172.21.15.70:0/366930360 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29e006bb20 0x7f29e006d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.314 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.312+0000 7f29e6158640 1 -- 172.21.15.70:0/366930360 >> v1:172.21.15.70:6789/0 conn(0x7f29e012dad0 legacy=0x7f29e012df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.312+0000 7f29e6158640 1 -- 172.21.15.70:0/366930360 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e006de30 con 0x7f29e006cb60 2024-09-13T15:43:40.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.313+0000 7f29e5957640 1 --2- 172.21.15.70:0/366930360 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29e006bb20 0x7f29e006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:43:40.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.314+0000 7f29e6158640 1 --2- 172.21.15.70:0/366930360 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e006cb60 0x7f29e006cf40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f29dc0097d0 tx=0x7f29dc02a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75564b7513906f6c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.314+0000 7f29e5156640 1 -- 172.21.15.70:0/366930360 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc02ad90 con 0x7f29e006cb60 2024-09-13T15:43:40.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.314+0000 7f29e5156640 1 -- 172.21.15.70:0/366930360 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc032440 con 0x7f29e006cb60 2024-09-13T15:43:40.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.314+0000 7f29e5156640 1 -- 172.21.15.70:0/366930360 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0327b0 con 0x7f29e006cb60 2024-09-13T15:43:40.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.315+0000 7f29e83bc640 1 -- 172.21.15.70:0/366930360 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e006cb60 msgr2=0x7f29e006cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.315+0000 7f29e83bc640 1 --2- 172.21.15.70:0/366930360 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e006cb60 0x7f29e006cf40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f29dc0097d0 tx=0x7f29dc02a040 comp rx=0 tx=0).stop 2024-09-13T15:43:40.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.316+0000 7f29e83bc640 1 -- 172.21.15.70:0/366930360 shutdown_connections 2024-09-13T15:43:40.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.316+0000 7f29e83bc640 1 --2- 172.21.15.70:0/366930360 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29e006bb20 0x7f29e006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.316+0000 7f29e83bc640 1 --2- 172.21.15.70:0/366930360 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e006cb60 0x7f29e006cf40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.316+0000 7f29e83bc640 1 -- 172.21.15.70:0/366930360 >> 172.21.15.70:0/366930360 conn(0x7f29e00973c0 msgr2=0x7f29e00977d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.316+0000 7f29e83bc640 1 -- 172.21.15.70:0/366930360 shutdown_connections 2024-09-13T15:43:40.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.316+0000 7f29e83bc640 1 -- 172.21.15.70:0/366930360 wait complete. 2024-09-13T15:43:40.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.317+0000 7f29e83bc640 1 Processor -- start 2024-09-13T15:43:40.318 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.317+0000 7f29e83bc640 1 -- start start 2024-09-13T15:43:40.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.318+0000 7f29e83bc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e006bb20 0x7f29e0131790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.318+0000 7f29e83bc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29e006cb60 0x7f29e0131cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.318+0000 7f29e83bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e012dad0 0x7f29e0132210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.318+0000 7f29e83bc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e00961f0 con 0x7f29e012dad0 2024-09-13T15:43:40.320 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.318+0000 7f29e83bc640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0134340 con 0x7f29e006bb20 2024-09-13T15:43:40.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.318+0000 7f29e83bc640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0071ad0 con 0x7f29e006cb60 2024-09-13T15:43:40.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.318+0000 7f29e5957640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29e006cb60 0x7f29e0131cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.318+0000 7f29e6959640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e012dad0 0x7f29e0132210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.319+0000 7f29e6158640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e006bb20 0x7f29e0131790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.319+0000 7f29e5957640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29e006cb60 0x7f29e0131cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:37952/0 (socket says 172.21.15.70:37952) 2024-09-13T15:43:40.321 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.319+0000 7f29e5957640 1 -- 172.21.15.70:0/2866646536 learned_addr learned my addr 172.21.15.70:0/2866646536 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:40.322 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.319+0000 7f29e6959640 1 -- 172.21.15.70:0/2866646536 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29e006cb60 msgr2=0x7f29e0131cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.320+0000 7f29e6959640 1 --2- 172.21.15.70:0/2866646536 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29e006cb60 0x7f29e0131cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.320+0000 7f29e6959640 1 -- 172.21.15.70:0/2866646536 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e006bb20 msgr2=0x7f29e0131790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.320+0000 7f29e6959640 1 --2- 172.21.15.70:0/2866646536 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e006bb20 0x7f29e0131790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.320+0000 7f29e6959640 1 -- 172.21.15.70:0/2866646536 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc009400 con 0x7f29e012dad0 2024-09-13T15:43:40.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.320+0000 7f29e5957640 1 --2- 172.21.15.70:0/2866646536 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29e006cb60 0x7f29e0131cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:40.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.320+0000 7f29e6158640 1 --2- 172.21.15.70:0/2866646536 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e006bb20 0x7f29e0131790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:40.323 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.320+0000 7f29e6959640 1 --2- 172.21.15.70:0/2866646536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e012dad0 0x7f29e0132210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f29d800ce20 tx=0x7f29d800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.324 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.321+0000 7f72961d4640 1 Processor -- start 2024-09-13T15:43:40.324 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.321+0000 7f72961d4640 1 -- start start 2024-09-13T15:43:40.324 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.321+0000 7f29d6ffd640 1 -- 172.21.15.70:0/2866646536 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d800ad40 con 0x7f29e012dad0 2024-09-13T15:43:40.324 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.321+0000 7f29d6ffd640 1 -- 172.21.15.70:0/2866646536 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d8012400 con 0x7f29e012dad0 2024-09-13T15:43:40.324 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.321+0000 7f29d6ffd640 1 -- 172.21.15.70:0/2866646536 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d800aea0 con 0x7f29e012dad0 2024-09-13T15:43:40.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.321+0000 7f72961d4640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f729006bab0 0x7f729006be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.321+0000 7f72961d4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f729006c3d0 0x7f7290131b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.321+0000 7f72961d4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729006e0d0 con 0x7f72901320a0 2024-09-13T15:43:40.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.321+0000 7f72961d4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729006ccc0 con 0x7f729006c3d0 2024-09-13T15:43:40.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.321+0000 7f72961d4640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7290095ed0 con 0x7f729006bab0 2024-09-13T15:43:40.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.322+0000 7f728effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f729006c3d0 0x7f7290131b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.322+0000 7f728effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f729006c3d0 0x7f7290131b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37220/0 (socket says 172.21.15.70:37220) 2024-09-13T15:43:40.325 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.322+0000 7f29e83bc640 1 -- 172.21.15.70:0/2866646536 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e01332b0 con 0x7f29e012dad0 2024-09-13T15:43:40.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.322+0000 7f728ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f72901320a0 0x7f7290134490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58130/0 (socket says 172.21.15.70:58130) 2024-09-13T15:43:40.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.322+0000 7f728effd640 1 -- 172.21.15.70:0/4029636244 learned_addr learned my addr 172.21.15.70:0/4029636244 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:40.326 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.322+0000 7f29e83bc640 1 -- 172.21.15.70:0/2866646536 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e0132930 con 0x7f29e012dad0 2024-09-13T15:43:40.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.322+0000 7f728f7fe640 1 --2- 172.21.15.70:0/4029636244 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f729006bab0 0x7f729006be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.322+0000 7f728effd640 1 -- 172.21.15.70:0/4029636244 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f729006bab0 msgr2=0x7f729006be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:40.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.323+0000 7f728effd640 1 --2- 172.21.15.70:0/4029636244 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f729006bab0 0x7f729006be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.323+0000 7f728effd640 1 -- 172.21.15.70:0/4029636244 >> v1:172.21.15.70:6789/0 conn(0x7f72901320a0 legacy=0x7f7290134490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.323+0000 7f728effd640 1 -- 172.21.15.70:0/4029636244 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7290099030 con 0x7f729006c3d0 2024-09-13T15:43:40.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.323+0000 7f728f7fe640 1 --2- 172.21.15.70:0/4029636244 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f729006bab0 0x7f729006be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:43:40.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.324+0000 7f29d6ffd640 1 -- 172.21.15.70:0/2866646536 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f29d8007070 con 0x7f29e012dad0 2024-09-13T15:43:40.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.324+0000 7f29e83bc640 1 -- 172.21.15.70:0/2866646536 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e006d790 con 0x7f29e012dad0 2024-09-13T15:43:40.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.324+0000 7f728effd640 1 --2- 172.21.15.70:0/4029636244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f729006c3d0 0x7f7290131b60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7284009870 tx=0x7f728402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfc088d8f3e538a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.326+0000 7f29e6158640 1 -- 172.21.15.70:0/2866646536 >> v1:172.21.15.70:6816/32242 conn(0x7f29c404d090 legacy=0x7f29c404f550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6816/32242 2024-09-13T15:43:40.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.326+0000 7f29d6ffd640 1 -- 172.21.15.70:0/2866646536 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f29d8063250 con 0x7f29e012dad0 2024-09-13T15:43:40.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.331+0000 7f29d6ffd640 1 -- 172.21.15.70:0/2866646536 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f29d803e030 con 0x7f29e012dad0 2024-09-13T15:43:40.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.331+0000 7f728e7fc640 1 -- 172.21.15.70:0/4029636244 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728402ae80 con 0x7f729006c3d0 2024-09-13T15:43:40.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.331+0000 7f728e7fc640 1 -- 172.21.15.70:0/4029636244 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7284032440 con 0x7f729006c3d0 2024-09-13T15:43:40.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.332+0000 7f728e7fc640 1 -- 172.21.15.70:0/4029636244 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728402ae80 con 0x7f729006c3d0 2024-09-13T15:43:40.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.332+0000 7f72961d4640 1 -- 172.21.15.70:0/4029636244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f729006c3d0 msgr2=0x7f7290131b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.332+0000 7f72961d4640 1 --2- 172.21.15.70:0/4029636244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f729006c3d0 0x7f7290131b60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7284009870 tx=0x7f728402a130 comp rx=0 tx=0).stop 2024-09-13T15:43:40.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.333+0000 7f72961d4640 1 -- 172.21.15.70:0/4029636244 shutdown_connections 2024-09-13T15:43:40.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.333+0000 7f72961d4640 1 --2- 172.21.15.70:0/4029636244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f729006c3d0 0x7f7290131b60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.333+0000 7f72961d4640 1 --2- 172.21.15.70:0/4029636244 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f729006bab0 0x7f729006be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.333+0000 7f72961d4640 1 -- 172.21.15.70:0/4029636244 >> 172.21.15.70:0/4029636244 conn(0x7f7290099cd0 msgr2=0x7f729009a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.333+0000 7f72961d4640 1 -- 172.21.15.70:0/4029636244 shutdown_connections 2024-09-13T15:43:40.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.333+0000 7f72961d4640 1 -- 172.21.15.70:0/4029636244 wait complete. 2024-09-13T15:43:40.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.334+0000 7f72961d4640 1 Processor -- start 2024-09-13T15:43:40.335 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.334+0000 7f72961d4640 1 -- start start 2024-09-13T15:43:40.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.335+0000 7f72961d4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f729006bab0 0x7f7290120a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.335+0000 7f72961d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f729006c3d0 0x7f7290120fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.335+0000 7f72961d4640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f72901320a0 0x7f72901214f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.335+0000 7f72961d4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7290121bd0 con 0x7f729006c3d0 2024-09-13T15:43:40.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.335+0000 7f72961d4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7290122640 con 0x7f729006bab0 2024-09-13T15:43:40.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.335+0000 7f72961d4640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7290071ac0 con 0x7f72901320a0 2024-09-13T15:43:40.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.335+0000 7f728f7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f729006bab0 0x7f7290120a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.335+0000 7f728f7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f729006bab0 0x7f7290120a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:54438/0 (socket says 172.21.15.70:54438) 2024-09-13T15:43:40.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.335+0000 7f728f7fe640 1 -- 172.21.15.70:0/401936517 learned_addr learned my addr 172.21.15.70:0/401936517 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:40.338 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:40.336+0000 7fd9b5c49040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T15:43:40.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.336+0000 7f728ffff640 1 --2- 172.21.15.70:0/401936517 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f72901320a0 0x7f72901214f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.336+0000 7f728f7fe640 1 -- 172.21.15.70:0/401936517 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f72901320a0 msgr2=0x7f72901214f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.336+0000 7f728f7fe640 1 --2- 172.21.15.70:0/401936517 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f72901320a0 0x7f72901214f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.336+0000 7f728f7fe640 1 -- 172.21.15.70:0/401936517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f729006c3d0 msgr2=0x7f7290120fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.336+0000 7f728f7fe640 1 --2- 172.21.15.70:0/401936517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f729006c3d0 0x7f7290120fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.336+0000 7f728f7fe640 1 -- 172.21.15.70:0/401936517 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72840094d0 con 0x7f729006bab0 2024-09-13T15:43:40.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.336+0000 7f728ffff640 1 --2- 172.21.15.70:0/401936517 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f72901320a0 0x7f72901214f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:43:40.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.337+0000 7f728f7fe640 1 --2- 172.21.15.70:0/401936517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f729006bab0 0x7f7290120a70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f728000c7f0 tx=0x7f728000cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.337+0000 7f7277fff640 1 -- 172.21.15.70:0/401936517 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72800079d0 con 0x7f729006bab0 2024-09-13T15:43:40.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.337+0000 7f7277fff640 1 -- 172.21.15.70:0/401936517 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7280007b30 con 0x7f729006bab0 2024-09-13T15:43:40.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.337+0000 7f7277fff640 1 -- 172.21.15.70:0/401936517 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728000ce40 con 0x7f729006bab0 2024-09-13T15:43:40.340 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:40.338+0000 7f1d6571d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T15:43:40.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.337+0000 7f72961d4640 1 -- 172.21.15.70:0/401936517 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729006c930 con 0x7f729006bab0 2024-09-13T15:43:40.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.337+0000 7f72961d4640 1 -- 172.21.15.70:0/401936517 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7290122e90 con 0x7f729006bab0 2024-09-13T15:43:40.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.339+0000 7f7277fff640 1 -- 172.21.15.70:0/401936517 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f72800113f0 con 0x7f729006bab0 2024-09-13T15:43:40.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.339+0000 7f72961d4640 1 -- 172.21.15.70:0/401936517 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729006be90 con 0x7f729006bab0 2024-09-13T15:43:40.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.340+0000 7f7277fff640 1 -- 172.21.15.70:0/401936517 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f7280063200 con 0x7f729006bab0 2024-09-13T15:43:40.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.340+0000 7f728effd640 1 -- 172.21.15.70:0/401936517 >> v1:172.21.15.70:6816/32242 conn(0x7f727c04d040 legacy=0x7f727c04f500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6816/32242 2024-09-13T15:43:40.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.346+0000 7f7a9a1ae640 1 Processor -- start 2024-09-13T15:43:40.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.346+0000 7f7a9a1ae640 1 -- start start 2024-09-13T15:43:40.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.346+0000 7f7a9a1ae640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7a9406cab0 0x7f7a94136d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.346+0000 7f7a9a1ae640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a94137240 0x7f7a94139630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.346+0000 7f7a937fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7a9406bfc0 0x7f7a9406c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58138/0 (socket says 172.21.15.70:58138) 2024-09-13T15:43:40.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.346+0000 7f7a92ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7a9406cab0 0x7f7a94136d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.346+0000 7f7a92ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7a9406cab0 0x7f7a94136d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37248/0 (socket says 172.21.15.70:37248) 2024-09-13T15:43:40.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.347+0000 7f7a9a1ae640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a9406de30 con 0x7f7a9406bfc0 2024-09-13T15:43:40.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.347+0000 7f7a9a1ae640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9406d9f0 con 0x7f7a9406cab0 2024-09-13T15:43:40.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.347+0000 7f7a9a1ae640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a94097e80 con 0x7f7a94137240 2024-09-13T15:43:40.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.347+0000 7f7a937fe640 1 -- 172.21.15.70:0/3646037286 learned_addr learned my addr 172.21.15.70:0/3646037286 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:40.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.347+0000 7f7a93fff640 1 --2- 172.21.15.70:0/3646037286 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a94137240 0x7f7a94139630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.347+0000 7f7a93fff640 1 -- 172.21.15.70:0/3646037286 >> v2:172.21.15.70:3300/0 conn(0x7f7a9406cab0 msgr2=0x7f7a94136d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:40.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.347+0000 7f7a93fff640 1 --2- 172.21.15.70:0/3646037286 >> v2:172.21.15.70:3300/0 conn(0x7f7a9406cab0 0x7f7a94136d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.347+0000 7f7a927fc640 1 -- 172.21.15.70:0/3646037286 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236900169 0 0) 0x7f7a9406de30 con 0x7f7a9406bfc0 2024-09-13T15:43:40.351 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.347+0000 7f7277fff640 1 -- 172.21.15.70:0/401936517 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f728003e0e0 con 0x7f729006bab0 2024-09-13T15:43:40.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.348+0000 7f7a93fff640 1 -- 172.21.15.70:0/3646037286 >> v1:172.21.15.70:6789/0 conn(0x7f7a9406bfc0 legacy=0x7f7a9406c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.348+0000 7f7a93fff640 1 -- 172.21.15.70:0/3646037286 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9406d280 con 0x7f7a94137240 2024-09-13T15:43:40.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.348+0000 7f7a92ffd640 1 --2- 172.21.15.70:0/3646037286 >> v2:172.21.15.70:3300/0 conn(0x7f7a9406cab0 0x7f7a94136d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:40.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.348+0000 7f7a93fff640 1 --2- 172.21.15.70:0/3646037286 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a94137240 0x7f7a94139630 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c009d90 tx=0x7f7a8c02a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4af07266b43dd38 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.349+0000 7f7a927fc640 1 -- 172.21.15.70:0/3646037286 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c007740 con 0x7f7a94137240 2024-09-13T15:43:40.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.349+0000 7f7a927fc640 1 -- 172.21.15.70:0/3646037286 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8c02d050 con 0x7f7a94137240 2024-09-13T15:43:40.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.349+0000 7f7a927fc640 1 -- 172.21.15.70:0/3646037286 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c007ac0 con 0x7f7a94137240 2024-09-13T15:43:40.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.349+0000 7f7a9a1ae640 1 -- 172.21.15.70:0/3646037286 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a94137240 msgr2=0x7f7a94139630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.349+0000 7f7a9a1ae640 1 --2- 172.21.15.70:0/3646037286 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a94137240 0x7f7a94139630 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c009d90 tx=0x7f7a8c02a530 comp rx=0 tx=0).stop 2024-09-13T15:43:40.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.350+0000 7f7a9a1ae640 1 -- 172.21.15.70:0/3646037286 shutdown_connections 2024-09-13T15:43:40.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.350+0000 7f7a9a1ae640 1 --2- 172.21.15.70:0/3646037286 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a94137240 0x7f7a94139630 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.350+0000 7f7a9a1ae640 1 --2- 172.21.15.70:0/3646037286 >> v2:172.21.15.70:3300/0 conn(0x7f7a9406cab0 0x7f7a94136d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.350+0000 7f7a9a1ae640 1 -- 172.21.15.70:0/3646037286 >> 172.21.15.70:0/3646037286 conn(0x7f7a9409bc00 msgr2=0x7f7a9409c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.350+0000 7f7a9a1ae640 1 -- 172.21.15.70:0/3646037286 shutdown_connections 2024-09-13T15:43:40.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.350+0000 7f7a9a1ae640 1 -- 172.21.15.70:0/3646037286 wait complete. 2024-09-13T15:43:40.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.351+0000 7f7a9a1ae640 1 Processor -- start 2024-09-13T15:43:40.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.351+0000 7f7a9a1ae640 1 -- start start 2024-09-13T15:43:40.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.351+0000 7f7a9a1ae640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9406bfc0 0x7f7a94134600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.352+0000 7f7a9a1ae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9406cab0 0x7f7a94134b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.352+0000 7f7a9a1ae640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7a94137240 0x7f7a941c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:40.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.352+0000 7f7a9a1ae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a94135740 con 0x7f7a9406cab0 2024-09-13T15:43:40.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.352+0000 7f7a9a1ae640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a94136530 con 0x7f7a9406bfc0 2024-09-13T15:43:40.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.352+0000 7f7a9a1ae640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a94135f40 con 0x7f7a94137240 2024-09-13T15:43:40.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.352+0000 7f7a92ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9406cab0 0x7f7a94134b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.352+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7a94137240 0x7f7a941c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.352+0000 7f7a92ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9406cab0 0x7f7a94134b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37252/0 (socket says 172.21.15.70:37252) 2024-09-13T15:43:40.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.352+0000 7f7a92ffd640 1 -- 172.21.15.70:0/2434063970 learned_addr learned my addr 172.21.15.70:0/2434063970 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:40.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.352+0000 7f7a937fe640 1 --2- 172.21.15.70:0/2434063970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9406bfc0 0x7f7a94134600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:40.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.353+0000 7f7a92ffd640 1 -- 172.21.15.70:0/2434063970 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7a94137240 msgr2=0x7f7a941c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.353+0000 7f7a92ffd640 1 --2- 172.21.15.70:0/2434063970 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7a94137240 0x7f7a941c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.353+0000 7f7a92ffd640 1 -- 172.21.15.70:0/2434063970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9406bfc0 msgr2=0x7f7a94134600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.353+0000 7f7a92ffd640 1 --2- 172.21.15.70:0/2434063970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9406bfc0 0x7f7a94134600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.353+0000 7f7a92ffd640 1 -- 172.21.15.70:0/2434063970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a88009400 con 0x7f7a9406cab0 2024-09-13T15:43:40.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.353+0000 7f7a92ffd640 1 --2- 172.21.15.70:0/2434063970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9406cab0 0x7f7a94134b40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800ec10 tx=0x7f7a8800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:40.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.353+0000 7f7a7bfff640 1 -- 172.21.15.70:0/2434063970 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88009760 con 0x7f7a9406cab0 2024-09-13T15:43:40.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.353+0000 7f7a7bfff640 1 -- 172.21.15.70:0/2434063970 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8800cc30 con 0x7f7a9406cab0 2024-09-13T15:43:40.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.353+0000 7f7a7bfff640 1 -- 172.21.15.70:0/2434063970 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88013470 con 0x7f7a9406cab0 2024-09-13T15:43:40.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.353+0000 7f7a9a1ae640 1 -- 172.21.15.70:0/2434063970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8c009a70 con 0x7f7a9406cab0 2024-09-13T15:43:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.354+0000 7f7a9a1ae640 1 -- 172.21.15.70:0/2434063970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a94122f30 con 0x7f7a9406cab0 2024-09-13T15:43:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.355+0000 7f7a7bfff640 1 -- 172.21.15.70:0/2434063970 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 6) v1 ==== 58690+0+0 (secure 0 0 0) 0x7f7a880135d0 con 0x7f7a9406cab0 2024-09-13T15:43:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.356+0000 7f7a7bfff640 1 -- 172.21.15.70:0/2434063970 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2836+0+0 (secure 0 0 0) 0x7f7a88064570 con 0x7f7a9406cab0 2024-09-13T15:43:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.356+0000 7f7a9a1ae640 1 -- 172.21.15.70:0/2434063970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a940008d0 con 0x7f7a9406cab0 2024-09-13T15:43:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.356+0000 7f7a937fe640 1 -- 172.21.15.70:0/2434063970 >> v1:172.21.15.70:6816/32242 conn(0x7f7a8004cfc0 legacy=0x7f7a8004f480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6816/32242 2024-09-13T15:43:40.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.360+0000 7f7a7bfff640 1 -- 172.21.15.70:0/2434063970 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f7a88017080 con 0x7f7a9406cab0 2024-09-13T15:43:40.428 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:40.427+0000 7f1d6571d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T15:43:40.429 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:40.427+0000 7fd9b5c49040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T15:43:40.432 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.431+0000 7fb063fff640 1 -- 172.21.15.70:0/3004522749 >> v1:172.21.15.70:6816/32242 conn(0x7fb05004cfc0 legacy=0x7fb05004f480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6816/32242 2024-09-13T15:43:40.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.501+0000 7f7a9a1ae640 1 -- 172.21.15.70:0/2434063970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7a94135080 con 0x7f7a9406cab0 2024-09-13T15:43:40.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.501+0000 7f7a7bfff640 1 -- 172.21.15.70:0/2434063970 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a8803dbe0 con 0x7f7a9406cab0 2024-09-13T15:43:40.503 INFO:teuthology.orchestra.run.smithi070.stdout:38654705681 2024-09-13T15:43:40.504 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.503+0000 7f7a9a1ae640 1 -- 172.21.15.70:0/2434063970 >> v1:172.21.15.70:6816/32242 conn(0x7f7a8004cfc0 legacy=0x7f7a8004f480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:43:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.503+0000 7f7a9a1ae640 1 -- 172.21.15.70:0/2434063970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9406cab0 msgr2=0x7f7a94134b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.503+0000 7f7a9a1ae640 1 --2- 172.21.15.70:0/2434063970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9406cab0 0x7f7a94134b40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800ec10 tx=0x7f7a8800c520 comp rx=0 tx=0).stop 2024-09-13T15:43:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.503+0000 7f7a9a1ae640 1 -- 172.21.15.70:0/2434063970 shutdown_connections 2024-09-13T15:43:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.503+0000 7f7a9a1ae640 1 --2- 172.21.15.70:0/2434063970 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7a94137240 0x7f7a941c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.503+0000 7f7a9a1ae640 1 --2- 172.21.15.70:0/2434063970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9406cab0 0x7f7a94134b40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.503+0000 7f7a9a1ae640 1 --2- 172.21.15.70:0/2434063970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9406bfc0 0x7f7a94134600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.503+0000 7f7a9a1ae640 1 -- 172.21.15.70:0/2434063970 >> 172.21.15.70:0/2434063970 conn(0x7f7a9409bc00 msgr2=0x7f7a94070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.503+0000 7f7a9a1ae640 1 -- 172.21.15.70:0/2434063970 shutdown_connections 2024-09-13T15:43:40.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.503+0000 7f7a9a1ae640 1 -- 172.21.15.70:0/2434063970 wait complete. 2024-09-13T15:43:40.515 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705681 for osd.3 2024-09-13T15:43:40.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.527+0000 7f29e6158640 1 -- 172.21.15.70:0/2866646536 >> v1:172.21.15.70:6816/32242 conn(0x7f29c404d090 legacy=0x7f29c404f550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6816/32242 2024-09-13T15:43:40.528 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.527+0000 7fb06aa5a640 1 -- 172.21.15.70:0/3004522749 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb06406a6b0 con 0x7fb06406ca10 2024-09-13T15:43:40.529 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.527+0000 7fb060ff9640 1 -- 172.21.15.70:0/3004522749 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb058017030 con 0x7fb06406ca10 2024-09-13T15:43:40.529 INFO:teuthology.orchestra.run.smithi070.stdout:38654705681 2024-09-13T15:43:40.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.530+0000 7fb06aa5a640 1 -- 172.21.15.70:0/3004522749 >> v1:172.21.15.70:6816/32242 conn(0x7fb05004cfc0 legacy=0x7fb05004f480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:43:40.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.530+0000 7fb06aa5a640 1 -- 172.21.15.70:0/3004522749 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb06406ca10 msgr2=0x7fb064127510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.530+0000 7fb06aa5a640 1 --2- 172.21.15.70:0/3004522749 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb06406ca10 0x7fb064127510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb05800c530 tx=0x7fb05800c9f0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.530+0000 7fb06aa5a640 1 -- 172.21.15.70:0/3004522749 shutdown_connections 2024-09-13T15:43:40.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.530+0000 7fb06aa5a640 1 --2- 172.21.15.70:0/3004522749 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb06412d960 0x7fb0641c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.530+0000 7fb06aa5a640 1 --2- 172.21.15.70:0/3004522749 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb06406ca10 0x7fb064127510 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.532 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.530+0000 7fb06aa5a640 1 --2- 172.21.15.70:0/3004522749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb06406c0f0 0x7fb064126fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.530+0000 7fb06aa5a640 1 -- 172.21.15.70:0/3004522749 >> 172.21.15.70:0/3004522749 conn(0x7fb0640a00d0 msgr2=0x7fb06409bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.531+0000 7fb06aa5a640 1 -- 172.21.15.70:0/3004522749 shutdown_connections 2024-09-13T15:43:40.533 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.531+0000 7fb06aa5a640 1 -- 172.21.15.70:0/3004522749 wait complete. 2024-09-13T15:43:40.542 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.541+0000 7f728effd640 1 -- 172.21.15.70:0/401936517 >> v1:172.21.15.70:6816/32242 conn(0x7f727c04d040 legacy=0x7f727c04f500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.70:6816/32242 2024-09-13T15:43:40.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705681 for osd.2 2024-09-13T15:43:40.557 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:40.556+0000 7f1d6571d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T15:43:40.572 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:40.571+0000 7fd9b5c49040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T15:43:40.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.604+0000 7f29e83bc640 1 -- 172.21.15.70:0/2866646536 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f29e006b330 con 0x7f29e012dad0 2024-09-13T15:43:40.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.605+0000 7f29d6ffd640 1 -- 172.21.15.70:0/2866646536 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f29d8017070 con 0x7f29e012dad0 2024-09-13T15:43:40.606 INFO:teuthology.orchestra.run.smithi070.stdout:38654705681 2024-09-13T15:43:40.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.606+0000 7f72961d4640 1 -- 172.21.15.70:0/401936517 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7290122e60 con 0x7f729006bab0 2024-09-13T15:43:40.607 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.606+0000 7f7277fff640 1 -- 172.21.15.70:0/401936517 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f72800418f0 con 0x7f729006bab0 2024-09-13T15:43:40.607 INFO:teuthology.orchestra.run.smithi070.stdout:38654705680 2024-09-13T15:43:40.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.607+0000 7f29e83bc640 1 -- 172.21.15.70:0/2866646536 >> v1:172.21.15.70:6816/32242 conn(0x7f29c404d090 legacy=0x7f29c404f550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:43:40.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.607+0000 7f29e83bc640 1 -- 172.21.15.70:0/2866646536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e012dad0 msgr2=0x7f29e0132210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.607+0000 7f29e83bc640 1 --2- 172.21.15.70:0/2866646536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e012dad0 0x7f29e0132210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f29d800ce20 tx=0x7f29d800a430 comp rx=0 tx=0).stop 2024-09-13T15:43:40.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.607+0000 7f29e83bc640 1 -- 172.21.15.70:0/2866646536 shutdown_connections 2024-09-13T15:43:40.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.607+0000 7f29e83bc640 1 --2- 172.21.15.70:0/2866646536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e012dad0 0x7f29e0132210 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.609 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.607+0000 7f29e83bc640 1 --2- 172.21.15.70:0/2866646536 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29e006cb60 0x7f29e0131cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.608+0000 7f29e83bc640 1 --2- 172.21.15.70:0/2866646536 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29e006bb20 0x7f29e0131790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.608+0000 7f29e83bc640 1 -- 172.21.15.70:0/2866646536 >> 172.21.15.70:0/2866646536 conn(0x7f29e00973c0 msgr2=0x7f29e009a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.608+0000 7f29e83bc640 1 -- 172.21.15.70:0/2866646536 shutdown_connections 2024-09-13T15:43:40.610 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.608+0000 7f29e83bc640 1 -- 172.21.15.70:0/2866646536 wait complete. 2024-09-13T15:43:40.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.608+0000 7f72961d4640 1 -- 172.21.15.70:0/401936517 >> v1:172.21.15.70:6816/32242 conn(0x7f727c04d040 legacy=0x7f727c04f500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:43:40.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.608+0000 7f72961d4640 1 -- 172.21.15.70:0/401936517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f729006bab0 msgr2=0x7f7290120a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:40.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.608+0000 7f72961d4640 1 --2- 172.21.15.70:0/401936517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f729006bab0 0x7f7290120a70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f728000c7f0 tx=0x7f728000cbf0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.608+0000 7f72961d4640 1 -- 172.21.15.70:0/401936517 shutdown_connections 2024-09-13T15:43:40.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.608+0000 7f72961d4640 1 --2- 172.21.15.70:0/401936517 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f72901320a0 0x7f72901214f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.608+0000 7f72961d4640 1 --2- 172.21.15.70:0/401936517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f729006c3d0 0x7f7290120fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.608+0000 7f72961d4640 1 --2- 172.21.15.70:0/401936517 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f729006bab0 0x7f7290120a70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:40.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.608+0000 7f72961d4640 1 -- 172.21.15.70:0/401936517 >> 172.21.15.70:0/401936517 conn(0x7f7290099cd0 msgr2=0x7f7290134490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:40.611 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.608+0000 7f72961d4640 1 -- 172.21.15.70:0/401936517 shutdown_connections 2024-09-13T15:43:40.612 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:40.608+0000 7f72961d4640 1 -- 172.21.15.70:0/401936517 wait complete. 2024-09-13T15:43:40.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705681 for osd.0 2024-09-13T15:43:40.626 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705681 got 38654705680 for osd.1 2024-09-13T15:43:40.661 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:40.660+0000 7f1d6571d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T15:43:40.673 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:40.672+0000 7fd9b5c49040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T15:43:40.709 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:40.708+0000 7f1d6571d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T15:43:40.717 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:40.715+0000 7fd9b5c49040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T15:43:40.811 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:40.810+0000 7f1d6571d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T15:43:40.816 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:40.815+0000 7fd9b5c49040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T15:43:40.865 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:40.864+0000 7f1d6571d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T15:43:40.869 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:40.868+0000 7fd9b5c49040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T15:43:40.911 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:40.910+0000 7f1d6571d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T15:43:40.915 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:40.914+0000 7fd9b5c49040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T15:43:41.024 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:41.023+0000 7f1d6571d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T15:43:41.025 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:41.024+0000 7fd9b5c49040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T15:43:41.078 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:41.077+0000 7f1d6571d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T15:43:41.078 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:41.077+0000 7fd9b5c49040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T15:43:41.131 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:41.130+0000 7f1d6571d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T15:43:41.131 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:41.130+0000 7fd9b5c49040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T15:43:41.201 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:41.200+0000 7fd9b5c49040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T15:43:41.202 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:41.201+0000 7f1d6571d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T15:43:41.247 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:41.246+0000 7fd9b5c49040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T15:43:41.248 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:41.247+0000 7f1d6571d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T15:43:41.293 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:41.292+0000 7fd9b5c49040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T15:43:41.293 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:41.292+0000 7f1d6571d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T15:43:41.516 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T15:43:41.526 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-09-13T15:43:41.525+0000 7fd9b5c49040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T15:43:41.526 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:41.525+0000 7f1d6571d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T15:43:41.543 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T15:43:41.626 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T15:43:41.626 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T15:43:41.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.626+0000 7f4327041640 1 Processor -- start 2024-09-13T15:43:41.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.626+0000 7f4327041640 1 -- start start 2024-09-13T15:43:41.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.627+0000 7f4327041640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f432006dd10 0x7f432012f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.627+0000 7f4327041640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f432006d570 0x7f4320131ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.627+0000 7f4327041640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4320097d40 con 0x7f432006ca80 2024-09-13T15:43:41.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.627+0000 7f4327041640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432006f1d0 con 0x7f432006dd10 2024-09-13T15:43:41.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.627+0000 7f4327041640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f432006ebc0 con 0x7f432006d570 2024-09-13T15:43:41.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.627+0000 7f431ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f432006dd10 0x7f432012f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.629 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.627+0000 7f43255de640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f432006d570 0x7f4320131ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.627+0000 7f431ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f432006dd10 0x7f432012f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37318/0 (socket says 172.21.15.70:37318) 2024-09-13T15:43:41.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.627+0000 7f4324ddd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f432006ca80 0x7f432006ce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58154/0 (socket says 172.21.15.70:58154) 2024-09-13T15:43:41.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.627+0000 7f431ffff640 1 -- 172.21.15.70:0/3500903979 learned_addr learned my addr 172.21.15.70:0/3500903979 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:41.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.627+0000 7f431ffff640 1 -- 172.21.15.70:0/3500903979 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f432006d570 msgr2=0x7f4320131ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:41.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.627+0000 7f431ffff640 1 --2- 172.21.15.70:0/3500903979 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f432006d570 0x7f4320131ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.627+0000 7f431ffff640 1 -- 172.21.15.70:0/3500903979 >> v1:172.21.15.70:6789/0 conn(0x7f432006ca80 legacy=0x7f432006ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.627+0000 7f431ffff640 1 -- 172.21.15.70:0/3500903979 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432006f400 con 0x7f432006dd10 2024-09-13T15:43:41.630 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.627+0000 7f431f7fe640 1 -- 172.21.15.70:0/3500903979 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814165584 0 0) 0x7f4320097d40 con 0x7f432006ca80 2024-09-13T15:43:41.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.628+0000 7f431ffff640 1 --2- 172.21.15.70:0/3500903979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432006dd10 0x7f432012f570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f430c00bb40 tx=0x7f430c02c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ebaef2e4a73505 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:41.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.628+0000 7f431f7fe640 1 -- 172.21.15.70:0/3500903979 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c0326f0 con 0x7f432006dd10 2024-09-13T15:43:41.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.628+0000 7f431f7fe640 1 -- 172.21.15.70:0/3500903979 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c032850 con 0x7f432006dd10 2024-09-13T15:43:41.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.628+0000 7f431f7fe640 1 -- 172.21.15.70:0/3500903979 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c032be0 con 0x7f432006dd10 2024-09-13T15:43:41.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.629+0000 7f4327041640 1 -- 172.21.15.70:0/3500903979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432006dd10 msgr2=0x7f432012f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.629+0000 7f4327041640 1 --2- 172.21.15.70:0/3500903979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432006dd10 0x7f432012f570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f430c00bb40 tx=0x7f430c02c830 comp rx=0 tx=0).stop 2024-09-13T15:43:41.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.629+0000 7f4327041640 1 -- 172.21.15.70:0/3500903979 shutdown_connections 2024-09-13T15:43:41.632 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.629+0000 7f4327041640 1 --2- 172.21.15.70:0/3500903979 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f432006d570 0x7f4320131ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.629+0000 7f4327041640 1 --2- 172.21.15.70:0/3500903979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432006dd10 0x7f432012f570 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.629+0000 7f4327041640 1 -- 172.21.15.70:0/3500903979 >> 172.21.15.70:0/3500903979 conn(0x7f4320098ae0 msgr2=0x7f43200963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:41.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.629+0000 7f4327041640 1 -- 172.21.15.70:0/3500903979 shutdown_connections 2024-09-13T15:43:41.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.629+0000 7f4327041640 1 -- 172.21.15.70:0/3500903979 wait complete. 2024-09-13T15:43:41.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.630+0000 7f4327041640 1 Processor -- start 2024-09-13T15:43:41.633 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.630+0000 7f4327041640 1 -- start start 2024-09-13T15:43:41.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.631+0000 7f4327041640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432006ca80 0x7f4320126470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.631+0000 7f4327041640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f432006d570 0x7f43201269b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.631+0000 7f4327041640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f432006dd10 0x7f4320126ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.635 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.631+0000 7f4327041640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432008bbc0 con 0x7f432006ca80 2024-09-13T15:43:41.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.631+0000 7f4327041640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4320071ca0 con 0x7f432006d570 2024-09-13T15:43:41.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.631+0000 7f4327041640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f432008cfb0 con 0x7f432006dd10 2024-09-13T15:43:41.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.632+0000 7f43255de640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f432006dd10 0x7f4320126ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.632+0000 7f431ffff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f432006d570 0x7f43201269b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.632+0000 7f43255de640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f432006dd10 0x7f4320126ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:38070/0 (socket says 172.21.15.70:38070) 2024-09-13T15:43:41.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.632+0000 7f43255de640 1 -- 172.21.15.70:0/1023622306 learned_addr learned my addr 172.21.15.70:0/1023622306 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:41.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.632+0000 7f4324ddd640 1 --2- 172.21.15.70:0/1023622306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432006ca80 0x7f4320126470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.632+0000 7f431ffff640 1 -- 172.21.15.70:0/1023622306 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f432006dd10 msgr2=0x7f4320126ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.632+0000 7f431ffff640 1 --2- 172.21.15.70:0/1023622306 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f432006dd10 0x7f4320126ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.632+0000 7f431ffff640 1 -- 172.21.15.70:0/1023622306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432006ca80 msgr2=0x7f4320126470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.636 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.632+0000 7f431ffff640 1 --2- 172.21.15.70:0/1023622306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432006ca80 0x7f4320126470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.632+0000 7f431ffff640 1 -- 172.21.15.70:0/1023622306 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c00b770 con 0x7f432006d570 2024-09-13T15:43:41.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.632+0000 7f43255de640 1 --2- 172.21.15.70:0/1023622306 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f432006dd10 0x7f4320126ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:41.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.632+0000 7f4324ddd640 1 --2- 172.21.15.70:0/1023622306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432006ca80 0x7f4320126470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:43:41.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.632+0000 7f431ffff640 1 --2- 172.21.15.70:0/1023622306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f432006d570 0x7f43201269b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f430c034000 tx=0x7f430c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:41.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.633+0000 7f431d7fa640 1 -- 172.21.15.70:0/1023622306 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c044070 con 0x7f432006d570 2024-09-13T15:43:41.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.633+0000 7f431d7fa640 1 -- 172.21.15.70:0/1023622306 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c040480 con 0x7f432006d570 2024-09-13T15:43:41.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.633+0000 7f431d7fa640 1 -- 172.21.15.70:0/1023622306 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c040850 con 0x7f432006d570 2024-09-13T15:43:41.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.633+0000 7f4327041640 1 -- 172.21.15.70:0/1023622306 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432008b2d0 con 0x7f432006d570 2024-09-13T15:43:41.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.633+0000 7f4327041640 1 -- 172.21.15.70:0/1023622306 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4320127610 con 0x7f432006d570 2024-09-13T15:43:41.637 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.634+0000 7f4327041640 1 -- 172.21.15.70:0/1023622306 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432006ce60 con 0x7f432006d570 2024-09-13T15:43:41.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.635+0000 7f431d7fa640 1 -- 172.21.15.70:0/1023622306 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 7) v1 ==== 56697+0+0 (secure 0 0 0) 0x7f430c035480 con 0x7f432006d570 2024-09-13T15:43:41.638 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.636+0000 7f431d7fa640 1 -- 172.21.15.70:0/1023622306 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f430c080b10 con 0x7f432006d570 2024-09-13T15:43:41.640 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.639+0000 7f431d7fa640 1 -- 172.21.15.70:0/1023622306 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f430c062440 con 0x7f432006d570 2024-09-13T15:43:41.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.647+0000 7fdf861c7640 1 Processor -- start 2024-09-13T15:43:41.648 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.647+0000 7fdf861c7640 1 -- start start 2024-09-13T15:43:41.649 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.647+0000 7fdf861c7640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf80091460 0x7fdf80091840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.648+0000 7fdf861c7640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdf8012aa60 0x7fdf801327c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.648+0000 7fdf861c7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf80093a10 con 0x7fdf80091d80 2024-09-13T15:43:41.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.648+0000 7fdf7effd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdf80091d80 0x7fdf8012a350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58156/0 (socket says 172.21.15.70:58156) 2024-09-13T15:43:41.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.648+0000 7fdf7effd640 1 -- 172.21.15.70:0/3367670281 learned_addr learned my addr 172.21.15.70:0/3367670281 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:41.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.648+0000 7fdf861c7640 1 -- 172.21.15.70:0/3367670281 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80066480 con 0x7fdf80091460 2024-09-13T15:43:41.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.648+0000 7fdf861c7640 1 -- 172.21.15.70:0/3367670281 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8009c090 con 0x7fdf8012aa60 2024-09-13T15:43:41.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.648+0000 7fdf7f7fe640 1 --2- 172.21.15.70:0/3367670281 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf80091460 0x7fdf80091840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.648+0000 7fdf7e7fc640 1 -- 172.21.15.70:0/3367670281 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309934851 0 0) 0x7fdf80093a10 con 0x7fdf80091d80 2024-09-13T15:43:41.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.648+0000 7fdf7ffff640 1 --2- 172.21.15.70:0/3367670281 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdf8012aa60 0x7fdf801327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.648+0000 7fdf7e7fc640 1 -- 172.21.15.70:0/3367670281 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf68003520 con 0x7fdf80091d80 2024-09-13T15:43:41.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.648+0000 7fdf7f7fe640 1 -- 172.21.15.70:0/3367670281 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdf8012aa60 msgr2=0x7fdf801327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:41.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.648+0000 7fdf7f7fe640 1 --2- 172.21.15.70:0/3367670281 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdf8012aa60 0x7fdf801327c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.650 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.648+0000 7fdf7f7fe640 1 -- 172.21.15.70:0/3367670281 >> v1:172.21.15.70:6789/0 conn(0x7fdf80091d80 legacy=0x7fdf8012a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.651 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.648+0000 7fdf7f7fe640 1 -- 172.21.15.70:0/3367670281 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf80093700 con 0x7fdf80091460 2024-09-13T15:43:41.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.649+0000 7fdf7f7fe640 1 --2- 172.21.15.70:0/3367670281 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf80091460 0x7fdf80091840 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdf700097d0 tx=0x7fdf7002a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d75e1fe76ba4797 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:41.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.650+0000 7fdf7e7fc640 1 -- 172.21.15.70:0/3367670281 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7002ad90 con 0x7fdf80091460 2024-09-13T15:43:41.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.650+0000 7fdf7e7fc640 1 -- 172.21.15.70:0/3367670281 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf70032440 con 0x7fdf80091460 2024-09-13T15:43:41.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.650+0000 7fdf7e7fc640 1 -- 172.21.15.70:0/3367670281 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf700327b0 con 0x7fdf80091460 2024-09-13T15:43:41.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.651+0000 7fdf861c7640 1 -- 172.21.15.70:0/3367670281 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf80091460 msgr2=0x7fdf80091840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.651+0000 7fdf861c7640 1 --2- 172.21.15.70:0/3367670281 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf80091460 0x7fdf80091840 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdf700097d0 tx=0x7fdf7002a040 comp rx=0 tx=0).stop 2024-09-13T15:43:41.652 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.651+0000 7fdf861c7640 1 -- 172.21.15.70:0/3367670281 shutdown_connections 2024-09-13T15:43:41.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.651+0000 7fdf861c7640 1 --2- 172.21.15.70:0/3367670281 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdf8012aa60 0x7fdf801327c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.651+0000 7fdf861c7640 1 --2- 172.21.15.70:0/3367670281 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf80091460 0x7fdf80091840 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.651+0000 7fdf861c7640 1 -- 172.21.15.70:0/3367670281 >> 172.21.15.70:0/3367670281 conn(0x7fdf8009c3e0 msgr2=0x7fdf8009c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:41.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.651+0000 7fdf861c7640 1 -- 172.21.15.70:0/3367670281 shutdown_connections 2024-09-13T15:43:41.653 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.651+0000 7fdf861c7640 1 -- 172.21.15.70:0/3367670281 wait complete. 2024-09-13T15:43:41.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.652+0000 7fdf861c7640 1 Processor -- start 2024-09-13T15:43:41.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.652+0000 7fdf861c7640 1 -- start start 2024-09-13T15:43:41.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.652+0000 7fdf861c7640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf80091d80 0x7fdf8006f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.654 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.652+0000 7fdf861c7640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdf8012aa60 0x7fdf8006fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.652+0000 7fdf861c7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf80070310 0x7fdf801c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.652+0000 7fdf861c7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80071040 con 0x7fdf80070310 2024-09-13T15:43:41.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.652+0000 7fdf861c7640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80071580 con 0x7fdf80091d80 2024-09-13T15:43:41.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.652+0000 7fdf861c7640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80072150 con 0x7fdf8012aa60 2024-09-13T15:43:41.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.653+0000 7fdf7effd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdf8012aa60 0x7fdf8006fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.653+0000 7fdf7effd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdf8012aa60 0x7fdf8006fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:38090/0 (socket says 172.21.15.70:38090) 2024-09-13T15:43:41.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.653+0000 7fdf7effd640 1 -- 172.21.15.70:0/880435099 learned_addr learned my addr 172.21.15.70:0/880435099 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:41.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.653+0000 7fdf7f7fe640 1 --2- 172.21.15.70:0/880435099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf80091d80 0x7fdf8006f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.653+0000 7fdf7ffff640 1 --2- 172.21.15.70:0/880435099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf80070310 0x7fdf801c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.653+0000 7fdf7ffff640 1 -- 172.21.15.70:0/880435099 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdf8012aa60 msgr2=0x7fdf8006fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.653+0000 7fdf7ffff640 1 --2- 172.21.15.70:0/880435099 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdf8012aa60 0x7fdf8006fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.653+0000 7fdf7ffff640 1 -- 172.21.15.70:0/880435099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf80091d80 msgr2=0x7fdf8006f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.655 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.653+0000 7fdf7ffff640 1 --2- 172.21.15.70:0/880435099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf80091d80 0x7fdf8006f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.653+0000 7fdf7ffff640 1 -- 172.21.15.70:0/880435099 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf70009400 con 0x7fdf80070310 2024-09-13T15:43:41.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.653+0000 7fdf7effd640 1 --2- 172.21.15.70:0/880435099 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdf8012aa60 0x7fdf8006fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:41.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.653+0000 7fdf7f7fe640 1 --2- 172.21.15.70:0/880435099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf80091d80 0x7fdf8006f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:41.656 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.653+0000 7fdf7ffff640 1 --2- 172.21.15.70:0/880435099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf80070310 0x7fdf801c45a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdf78007d90 tx=0x7fdf7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:41.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.654+0000 7fdf67fff640 1 -- 172.21.15.70:0/880435099 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78018070 con 0x7fdf80070310 2024-09-13T15:43:41.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.654+0000 7fdf67fff640 1 -- 172.21.15.70:0/880435099 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7800edd0 con 0x7fdf80070310 2024-09-13T15:43:41.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.654+0000 7fdf67fff640 1 -- 172.21.15.70:0/880435099 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78003bb0 con 0x7fdf80070310 2024-09-13T15:43:41.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.654+0000 7fdf861c7640 1 -- 172.21.15.70:0/880435099 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf800707c0 con 0x7fdf80070310 2024-09-13T15:43:41.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.654+0000 7fdf861c7640 1 -- 172.21.15.70:0/880435099 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf80071930 con 0x7fdf80070310 2024-09-13T15:43:41.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.655+0000 7fdf67fff640 1 -- 172.21.15.70:0/880435099 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 7) v1 ==== 56697+0+0 (secure 0 0 0) 0x7fdf78014420 con 0x7fdf80070310 2024-09-13T15:43:41.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.655+0000 7fdf861c7640 1 -- 172.21.15.70:0/880435099 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf80121b30 con 0x7fdf80070310 2024-09-13T15:43:41.657 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.656+0000 7fdf67fff640 1 -- 172.21.15.70:0/880435099 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fdf7805fdc0 con 0x7fdf80070310 2024-09-13T15:43:41.660 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.659+0000 7fdf67fff640 1 -- 172.21.15.70:0/880435099 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7fdf7803dbc0 con 0x7fdf80070310 2024-09-13T15:43:41.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.728+0000 7f5294672640 1 Processor -- start 2024-09-13T15:43:41.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.728+0000 7f5294672640 1 -- start start 2024-09-13T15:43:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.728+0000 7f5294672640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f528c12d330 0x7f528c12d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.728+0000 7f5294672640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f528c12a540 0x7f528c135670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.729+0000 7f5294672640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528c06d5d0 con 0x7f528c12dd80 2024-09-13T15:43:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.729+0000 7f5294672640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f528c0971b0 con 0x7f528c12a540 2024-09-13T15:43:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.729+0000 7f5294672640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0979f0 con 0x7f528c12d330 2024-09-13T15:43:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.729+0000 7f5291c0d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f528c12dd80 0x7f528c129e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58166/0 (socket says 172.21.15.70:58166) 2024-09-13T15:43:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.729+0000 7f5291c0d640 1 -- 172.21.15.70:0/3412196191 learned_addr learned my addr 172.21.15.70:0/3412196191 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.729+0000 7f529240e640 1 --2- 172.21.15.70:0/3412196191 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f528c12d330 0x7f528c12d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.729+0000 7f5292c0f640 1 --2- 172.21.15.70:0/3412196191 >> v2:172.21.15.70:3300/0 conn(0x7f528c12a540 0x7f528c135670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.729+0000 7f529140c640 1 -- 172.21.15.70:0/3412196191 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630677626 0 0) 0x7f528c06d5d0 con 0x7f528c12dd80 2024-09-13T15:43:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.729+0000 7f529140c640 1 -- 172.21.15.70:0/3412196191 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5278003520 con 0x7f528c12dd80 2024-09-13T15:43:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.729+0000 7f529240e640 1 -- 172.21.15.70:0/3412196191 >> v2:172.21.15.70:3300/0 conn(0x7f528c12a540 msgr2=0x7f528c135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:41.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.729+0000 7f529240e640 1 --2- 172.21.15.70:0/3412196191 >> v2:172.21.15.70:3300/0 conn(0x7f528c12a540 0x7f528c135670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.729+0000 7f529240e640 1 -- 172.21.15.70:0/3412196191 >> v1:172.21.15.70:6789/0 conn(0x7f528c12dd80 legacy=0x7f528c129e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.729+0000 7f529240e640 1 -- 172.21.15.70:0/3412196191 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c12e1a0 con 0x7f528c12d330 2024-09-13T15:43:41.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.730+0000 7f529240e640 1 --2- 172.21.15.70:0/3412196191 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f528c12d330 0x7f528c12d840 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f52880097a0 tx=0x7f528802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=985ef76ca41272c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:41.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.730+0000 7f529140c640 1 -- 172.21.15.70:0/3412196191 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528802adb0 con 0x7f528c12d330 2024-09-13T15:43:41.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.731+0000 7f529140c640 1 -- 172.21.15.70:0/3412196191 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5288032440 con 0x7f528c12d330 2024-09-13T15:43:41.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.731+0000 7f529140c640 1 -- 172.21.15.70:0/3412196191 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880327d0 con 0x7f528c12d330 2024-09-13T15:43:41.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.731+0000 7f5294672640 1 -- 172.21.15.70:0/3412196191 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f528c12d330 msgr2=0x7f528c12d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.731+0000 7f5294672640 1 --2- 172.21.15.70:0/3412196191 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f528c12d330 0x7f528c12d840 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f52880097a0 tx=0x7f528802a060 comp rx=0 tx=0).stop 2024-09-13T15:43:41.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.731+0000 7f5294672640 1 -- 172.21.15.70:0/3412196191 shutdown_connections 2024-09-13T15:43:41.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.731+0000 7f5294672640 1 --2- 172.21.15.70:0/3412196191 >> v2:172.21.15.70:3300/0 conn(0x7f528c12a540 0x7f528c135670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.731+0000 7f5294672640 1 --2- 172.21.15.70:0/3412196191 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f528c12d330 0x7f528c12d840 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.731+0000 7f5294672640 1 -- 172.21.15.70:0/3412196191 >> 172.21.15.70:0/3412196191 conn(0x7f528c0a0060 msgr2=0x7f528c0a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:41.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.731+0000 7f5294672640 1 -- 172.21.15.70:0/3412196191 shutdown_connections 2024-09-13T15:43:41.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.731+0000 7f5294672640 1 -- 172.21.15.70:0/3412196191 wait complete. 2024-09-13T15:43:41.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.732+0000 7f5294672640 1 Processor -- start 2024-09-13T15:43:41.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.732+0000 7f5294672640 1 -- start start 2024-09-13T15:43:41.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5294672640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f528c12a540 0x7f528c12ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5294672640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f528c12d330 0x7f528c12f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5294672640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f528c12dd80 0x7f528c12f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5294672640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c1312c0 con 0x7f528c12dd80 2024-09-13T15:43:41.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5294672640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c131630 con 0x7f528c12a540 2024-09-13T15:43:41.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5294672640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f528c130520 con 0x7f528c12d330 2024-09-13T15:43:41.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f529240e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f528c12a540 0x7f528c12ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5291c0d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f528c12d330 0x7f528c12f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5292c0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f528c12dd80 0x7f528c12f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5291c0d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f528c12d330 0x7f528c12f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:38104/0 (socket says 172.21.15.70:38104) 2024-09-13T15:43:41.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5291c0d640 1 -- 172.21.15.70:0/4207221780 learned_addr learned my addr 172.21.15.70:0/4207221780 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:41.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5291c0d640 1 -- 172.21.15.70:0/4207221780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f528c12a540 msgr2=0x7f528c12ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5291c0d640 1 --2- 172.21.15.70:0/4207221780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f528c12a540 0x7f528c12ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5291c0d640 1 -- 172.21.15.70:0/4207221780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f528c12dd80 msgr2=0x7f528c12f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5291c0d640 1 --2- 172.21.15.70:0/4207221780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f528c12dd80 0x7f528c12f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5291c0d640 1 -- 172.21.15.70:0/4207221780 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5288009400 con 0x7f528c12d330 2024-09-13T15:43:41.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f5292c0f640 1 --2- 172.21.15.70:0/4207221780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f528c12dd80 0x7f528c12f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:41.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.733+0000 7f529240e640 1 --2- 172.21.15.70:0/4207221780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f528c12a540 0x7f528c12ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:41.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.734+0000 7f5291c0d640 1 --2- 172.21.15.70:0/4207221780 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f528c12d330 0x7f528c12f1d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f527c00c500 tx=0x7f527c00c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:41.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.734+0000 7f5282ffd640 1 -- 172.21.15.70:0/4207221780 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c005ce0 con 0x7f528c12d330 2024-09-13T15:43:41.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.734+0000 7f5282ffd640 1 -- 172.21.15.70:0/4207221780 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527c005e40 con 0x7f528c12d330 2024-09-13T15:43:41.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.734+0000 7f5282ffd640 1 -- 172.21.15.70:0/4207221780 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c007330 con 0x7f528c12d330 2024-09-13T15:43:41.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.734+0000 7f5294672640 1 -- 172.21.15.70:0/4207221780 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528c131e10 con 0x7f528c12d330 2024-09-13T15:43:41.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.734+0000 7f5294672640 1 -- 172.21.15.70:0/4207221780 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c130f80 con 0x7f528c12d330 2024-09-13T15:43:41.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.735+0000 7f5282ffd640 1 -- 172.21.15.70:0/4207221780 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 7) v1 ==== 56697+0+0 (secure 0 0 0) 0x7f527c007490 con 0x7f528c12d330 2024-09-13T15:43:41.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.735+0000 7f5294672640 1 -- 172.21.15.70:0/4207221780 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528c12d840 con 0x7f528c12d330 2024-09-13T15:43:41.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.736+0000 7f5282ffd640 1 -- 172.21.15.70:0/4207221780 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f527c05f130 con 0x7f528c12d330 2024-09-13T15:43:41.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.742+0000 7f5282ffd640 1 -- 172.21.15.70:0/4207221780 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7f527c03d070 con 0x7f528c12d330 2024-09-13T15:43:41.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.746+0000 7feeba880640 1 Processor -- start 2024-09-13T15:43:41.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeba880640 1 -- start start 2024-09-13T15:43:41.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeba880640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeb40953c0 0x7feeb40957a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeba880640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feeb406b970 0x7feeb406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeba880640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feeb406c330 0x7feeb4138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeba880640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb406e050 con 0x7feeb406b970 2024-09-13T15:43:41.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeba880640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb406da90 con 0x7feeb40953c0 2024-09-13T15:43:41.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeba880640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feeb4097d40 con 0x7feeb406c330 2024-09-13T15:43:41.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeb8e1d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feeb406c330 0x7feeb4138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeb8e1d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feeb406c330 0x7feeb4138ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:38114/0 (socket says 172.21.15.70:38114) 2024-09-13T15:43:41.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeb8e1d640 1 -- 172.21.15.70:0/3888147785 learned_addr learned my addr 172.21.15.70:0/3888147785 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeb40953c0 0x7feeb40957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeb37fe640 1 --2- 172.21.15.70:0/3888147785 >> v2:172.21.15.70:3300/0 conn(0x7feeb406b970 0x7feeb406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeb8e1d640 1 -- 172.21.15.70:0/3888147785 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeb40953c0 msgr2=0x7feeb40957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeb8e1d640 1 --2- 172.21.15.70:0/3888147785 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeb40953c0 0x7feeb40957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeb8e1d640 1 -- 172.21.15.70:0/3888147785 >> v2:172.21.15.70:3300/0 conn(0x7feeb406b970 msgr2=0x7feeb406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.747+0000 7feeb8e1d640 1 --2- 172.21.15.70:0/3888147785 >> v2:172.21.15.70:3300/0 conn(0x7feeb406b970 0x7feeb406bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.748+0000 7feeb8e1d640 1 -- 172.21.15.70:0/3888147785 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb406c780 con 0x7feeb406c330 2024-09-13T15:43:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.748+0000 7feeb3fff640 1 --2- 172.21.15.70:0/3888147785 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeb40953c0 0x7feeb40957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.748+0000 7feeb37fe640 1 --2- 172.21.15.70:0/3888147785 >> v2:172.21.15.70:3300/0 conn(0x7feeb406b970 0x7feeb406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.748+0000 7feeb8e1d640 1 --2- 172.21.15.70:0/3888147785 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feeb406c330 0x7feeb4138ab0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feeac009e80 tx=0x7feeac02a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d9320aa763f62d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.749+0000 7feeb2ffd640 1 -- 172.21.15.70:0/3888147785 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac031400 con 0x7feeb406c330 2024-09-13T15:43:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.749+0000 7feeb2ffd640 1 -- 172.21.15.70:0/3888147785 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeac02d050 con 0x7feeb406c330 2024-09-13T15:43:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.749+0000 7feeb2ffd640 1 -- 172.21.15.70:0/3888147785 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac031780 con 0x7feeb406c330 2024-09-13T15:43:41.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.749+0000 7feeba880640 1 -- 172.21.15.70:0/3888147785 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feeb406c330 msgr2=0x7feeb4138ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.749+0000 7feeba880640 1 --2- 172.21.15.70:0/3888147785 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feeb406c330 0x7feeb4138ab0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feeac009e80 tx=0x7feeac02a620 comp rx=0 tx=0).stop 2024-09-13T15:43:41.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.750+0000 7feeba880640 1 -- 172.21.15.70:0/3888147785 shutdown_connections 2024-09-13T15:43:41.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.750+0000 7feeba880640 1 --2- 172.21.15.70:0/3888147785 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feeb406c330 0x7feeb4138ab0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.750+0000 7feeba880640 1 --2- 172.21.15.70:0/3888147785 >> v2:172.21.15.70:3300/0 conn(0x7feeb406b970 0x7feeb406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.750+0000 7feeba880640 1 --2- 172.21.15.70:0/3888147785 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeb40953c0 0x7feeb40957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.750+0000 7feeba880640 1 -- 172.21.15.70:0/3888147785 >> 172.21.15.70:0/3888147785 conn(0x7feeb4098ae0 msgr2=0x7feeb40963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:41.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.750+0000 7feeba880640 1 -- 172.21.15.70:0/3888147785 shutdown_connections 2024-09-13T15:43:41.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.750+0000 7feeba880640 1 -- 172.21.15.70:0/3888147785 wait complete. 2024-09-13T15:43:41.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.751+0000 7feeba880640 1 Processor -- start 2024-09-13T15:43:41.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.751+0000 7feeba880640 1 -- start start 2024-09-13T15:43:41.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.752+0000 7feeba880640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb406b970 0x7feeb41c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.752+0000 7feeba880640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeb406c330 0x7feeb41c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.752+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb406b970 0x7feeb41c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.752+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb406b970 0x7feeb41c66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37374/0 (socket says 172.21.15.70:37374) 2024-09-13T15:43:41.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.752+0000 7feeb37fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeb406c330 0x7feeb41c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.752+0000 7feeb37fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeb406c330 0x7feeb41c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:54574/0 (socket says 172.21.15.70:54574) 2024-09-13T15:43:41.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.752+0000 7feeb3fff640 1 -- 172.21.15.70:0/3778639036 learned_addr learned my addr 172.21.15.70:0/3778639036 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:41.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.752+0000 7feeba880640 1 --2- 172.21.15.70:0/3778639036 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feeb40953c0 0x7feeb41c7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:41.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.752+0000 7feeba880640 1 -- 172.21.15.70:0/3778639036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb412e5f0 con 0x7feeb406b970 2024-09-13T15:43:41.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.752+0000 7feeba880640 1 -- 172.21.15.70:0/3778639036 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb408c180 con 0x7feeb406c330 2024-09-13T15:43:41.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.752+0000 7feeba880640 1 -- 172.21.15.70:0/3778639036 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feeb408cbc0 con 0x7feeb40953c0 2024-09-13T15:43:41.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.752+0000 7feeb8e1d640 1 --2- 172.21.15.70:0/3778639036 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feeb40953c0 0x7feeb41c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:41.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.753+0000 7feeb8e1d640 1 -- 172.21.15.70:0/3778639036 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeb406c330 msgr2=0x7feeb41c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.753+0000 7feeb8e1d640 1 --2- 172.21.15.70:0/3778639036 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeb406c330 0x7feeb41c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.753+0000 7feeb8e1d640 1 -- 172.21.15.70:0/3778639036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb406b970 msgr2=0x7feeb41c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.753+0000 7feeb8e1d640 1 --2- 172.21.15.70:0/3778639036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb406b970 0x7feeb41c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.753+0000 7feeb8e1d640 1 -- 172.21.15.70:0/3778639036 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea8009400 con 0x7feeb40953c0 2024-09-13T15:43:41.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.753+0000 7feeb37fe640 1 --2- 172.21.15.70:0/3778639036 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeb406c330 0x7feeb41c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:41.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.753+0000 7feeb8e1d640 1 --2- 172.21.15.70:0/3778639036 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feeb40953c0 0x7feeb41c7140 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7feeac030000 tx=0x7feeac004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:41.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.753+0000 7feeb0ff9640 1 -- 172.21.15.70:0/3778639036 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac03d650 con 0x7feeb40953c0 2024-09-13T15:43:41.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.753+0000 7feeb0ff9640 1 -- 172.21.15.70:0/3778639036 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeac03d7d0 con 0x7feeb40953c0 2024-09-13T15:43:41.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.753+0000 7feeb0ff9640 1 -- 172.21.15.70:0/3778639036 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac03db80 con 0x7feeb40953c0 2024-09-13T15:43:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.753+0000 7feeba880640 1 -- 172.21.15.70:0/3778639036 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea4009400 con 0x7feeb40953c0 2024-09-13T15:43:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.753+0000 7feeba880640 1 -- 172.21.15.70:0/3778639036 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeac009b30 con 0x7feeb40953c0 2024-09-13T15:43:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.754+0000 7feeb0ff9640 1 -- 172.21.15.70:0/3778639036 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 7) v1 ==== 56697+0+0 (secure 0 0 0) 0x7feeac03dce0 con 0x7feeb40953c0 2024-09-13T15:43:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.755+0000 7feeb0ff9640 1 -- 172.21.15.70:0/3778639036 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3023+0+0 (secure 0 0 0) 0x7feeac080500 con 0x7feeb40953c0 2024-09-13T15:43:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.755+0000 7feeba880640 1 -- 172.21.15.70:0/3778639036 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb406bdf0 con 0x7feeb40953c0 2024-09-13T15:43:41.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.758+0000 7feeb0ff9640 1 -- 172.21.15.70:0/3778639036 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+141550 (secure 0 0 0) 0x7feeac036080 con 0x7feeb40953c0 2024-09-13T15:43:41.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.803+0000 7fdf861c7640 1 -- 172.21.15.70:0/880435099 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdf80091840 con 0x7fdf80070310 2024-09-13T15:43:41.804 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.803+0000 7fdf67fff640 1 -- 172.21.15.70:0/880435099 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdf780411e0 con 0x7fdf80070310 2024-09-13T15:43:41.805 INFO:teuthology.orchestra.run.smithi070.stdout:38654705681 2024-09-13T15:43:41.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.805+0000 7fdf861c7640 1 -- 172.21.15.70:0/880435099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf80070310 msgr2=0x7fdf801c45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.805+0000 7fdf861c7640 1 --2- 172.21.15.70:0/880435099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf80070310 0x7fdf801c45a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdf78007d90 tx=0x7fdf7800e4b0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.806 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.805+0000 7fdf861c7640 1 -- 172.21.15.70:0/880435099 shutdown_connections 2024-09-13T15:43:41.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.805+0000 7fdf861c7640 1 --2- 172.21.15.70:0/880435099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf80070310 0x7fdf801c45a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.805+0000 7fdf861c7640 1 --2- 172.21.15.70:0/880435099 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdf8012aa60 0x7fdf8006fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.805+0000 7fdf861c7640 1 --2- 172.21.15.70:0/880435099 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf80091d80 0x7fdf8006f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.805+0000 7fdf861c7640 1 -- 172.21.15.70:0/880435099 >> 172.21.15.70:0/880435099 conn(0x7fdf8009c3e0 msgr2=0x7fdf800953e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:41.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.805+0000 7fdf861c7640 1 -- 172.21.15.70:0/880435099 shutdown_connections 2024-09-13T15:43:41.807 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.805+0000 7fdf861c7640 1 -- 172.21.15.70:0/880435099 wait complete. 2024-09-13T15:43:41.817 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705681 for osd.2 2024-09-13T15:43:41.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.822+0000 7f4327041640 1 -- 172.21.15.70:0/1023622306 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4320122180 con 0x7f432006d570 2024-09-13T15:43:41.824 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.823+0000 7f431d7fa640 1 -- 172.21.15.70:0/1023622306 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f430c062030 con 0x7f432006d570 2024-09-13T15:43:41.824 INFO:teuthology.orchestra.run.smithi070.stdout:38654705681 2024-09-13T15:43:41.825 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.824+0000 7f4327041640 1 -- 172.21.15.70:0/1023622306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f432006d570 msgr2=0x7f43201269b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.824+0000 7f4327041640 1 --2- 172.21.15.70:0/1023622306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f432006d570 0x7f43201269b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f430c034000 tx=0x7f430c004730 comp rx=0 tx=0).stop 2024-09-13T15:43:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.824+0000 7f4327041640 1 -- 172.21.15.70:0/1023622306 shutdown_connections 2024-09-13T15:43:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.824+0000 7f4327041640 1 --2- 172.21.15.70:0/1023622306 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f432006dd10 0x7f4320126ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.824+0000 7f4327041640 1 --2- 172.21.15.70:0/1023622306 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f432006d570 0x7f43201269b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.824+0000 7f4327041640 1 --2- 172.21.15.70:0/1023622306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f432006ca80 0x7f4320126470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.824+0000 7f4327041640 1 -- 172.21.15.70:0/1023622306 >> 172.21.15.70:0/1023622306 conn(0x7f4320098ae0 msgr2=0x7f432009a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.824+0000 7f4327041640 1 -- 172.21.15.70:0/1023622306 shutdown_connections 2024-09-13T15:43:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.824+0000 7f4327041640 1 -- 172.21.15.70:0/1023622306 wait complete. 2024-09-13T15:43:41.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705681 for osd.3 2024-09-13T15:43:41.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.884+0000 7f5294672640 1 -- 172.21.15.70:0/4207221780 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f528c126410 con 0x7f528c12d330 2024-09-13T15:43:41.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.884+0000 7f5282ffd640 1 -- 172.21.15.70:0/4207221780 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f527c017030 con 0x7f528c12d330 2024-09-13T15:43:41.886 INFO:teuthology.orchestra.run.smithi070.stdout:38654705681 2024-09-13T15:43:41.887 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.886+0000 7f5294672640 1 -- 172.21.15.70:0/4207221780 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f528c12d330 msgr2=0x7f528c12f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.886+0000 7f5294672640 1 --2- 172.21.15.70:0/4207221780 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f528c12d330 0x7f528c12f1d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f527c00c500 tx=0x7f527c00c9c0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.886+0000 7f5294672640 1 -- 172.21.15.70:0/4207221780 shutdown_connections 2024-09-13T15:43:41.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.886+0000 7f5294672640 1 --2- 172.21.15.70:0/4207221780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f528c12dd80 0x7f528c12f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.886+0000 7f5294672640 1 --2- 172.21.15.70:0/4207221780 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f528c12d330 0x7f528c12f1d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.886+0000 7f5294672640 1 --2- 172.21.15.70:0/4207221780 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f528c12a540 0x7f528c12ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.886+0000 7f5294672640 1 -- 172.21.15.70:0/4207221780 >> 172.21.15.70:0/4207221780 conn(0x7f528c0a0060 msgr2=0x7f528c0984d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:41.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.886+0000 7f5294672640 1 -- 172.21.15.70:0/4207221780 shutdown_connections 2024-09-13T15:43:41.888 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.886+0000 7f5294672640 1 -- 172.21.15.70:0/4207221780 wait complete. 2024-09-13T15:43:41.898 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705681 for osd.0 2024-09-13T15:43:41.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.914+0000 7feeba880640 1 -- 172.21.15.70:0/3778639036 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7feeb4066fa0 con 0x7feeb40953c0 2024-09-13T15:43:41.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.914+0000 7feeb0ff9640 1 -- 172.21.15.70:0/3778639036 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feeac041020 con 0x7feeb40953c0 2024-09-13T15:43:41.916 INFO:teuthology.orchestra.run.smithi070.stdout:38654705680 2024-09-13T15:43:41.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.916+0000 7feeba880640 1 -- 172.21.15.70:0/3778639036 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feeb40953c0 msgr2=0x7feeb41c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:41.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.916+0000 7feeba880640 1 --2- 172.21.15.70:0/3778639036 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feeb40953c0 0x7feeb41c7140 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7feeac030000 tx=0x7feeac004a40 comp rx=0 tx=0).stop 2024-09-13T15:43:41.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.916+0000 7feeba880640 1 -- 172.21.15.70:0/3778639036 shutdown_connections 2024-09-13T15:43:41.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.916+0000 7feeba880640 1 --2- 172.21.15.70:0/3778639036 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feeb40953c0 0x7feeb41c7140 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.916+0000 7feeba880640 1 --2- 172.21.15.70:0/3778639036 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feeb406c330 0x7feeb41c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.916+0000 7feeba880640 1 --2- 172.21.15.70:0/3778639036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb406b970 0x7feeb41c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:41.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.916+0000 7feeba880640 1 -- 172.21.15.70:0/3778639036 >> 172.21.15.70:0/3778639036 conn(0x7feeb4098ae0 msgr2=0x7feeb4131570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:41.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.916+0000 7feeba880640 1 -- 172.21.15.70:0/3778639036 shutdown_connections 2024-09-13T15:43:41.918 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:41.916+0000 7feeba880640 1 -- 172.21.15.70:0/3778639036 wait complete. 2024-09-13T15:43:41.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705681 got 38654705680 for osd.1 2024-09-13T15:43:42.536 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:42.535+0000 7f1d4ef08640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-13T15:43:42.537 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:42.536+0000 7f1d4ef08640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T15:43:42.537 INFO:tasks.ceph.mgr.y.smithi070.stderr:2024-09-13T15:43:42.536+0000 7f1d4ef08640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T15:43:42.818 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T15:43:42.837 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T15:43:42.899 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T15:43:42.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.926+0000 7f96f8725640 1 Processor -- start 2024-09-13T15:43:42.928 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.926+0000 7f96f8725640 1 -- start start 2024-09-13T15:43:42.928 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T15:43:42.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.927+0000 7f96f8725640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f96f0070c30 0x7f96f0071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:42.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.927+0000 7f96f8725640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f96f012b870 0x7f96f012bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:42.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.927+0000 7f96f8725640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f0070970 con 0x7f96f0071550 2024-09-13T15:43:42.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.927+0000 7f96f6cc2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f96f012b870 0x7f96f012bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:42.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.927+0000 7f96f6cc2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f96f012b870 0x7f96f012bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37378/0 (socket says 172.21.15.70:37378) 2024-09-13T15:43:42.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.927+0000 7f96f6cc2640 1 -- 172.21.15.70:0/843804749 learned_addr learned my addr 172.21.15.70:0/843804749 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:42.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.927+0000 7f96f8725640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f0070130 con 0x7f96f012b870 2024-09-13T15:43:42.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.927+0000 7f96f8725640 1 -- 172.21.15.70:0/843804749 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96f0071ce0 con 0x7f96f0070c30 2024-09-13T15:43:42.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.927+0000 7f96f64c1640 1 --2- 172.21.15.70:0/843804749 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f96f0070c30 0x7f96f0071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:42.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.927+0000 7f96f6cc2640 1 -- 172.21.15.70:0/843804749 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f96f0070c30 msgr2=0x7f96f0071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:42.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.927+0000 7f96f6cc2640 1 --2- 172.21.15.70:0/843804749 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f96f0070c30 0x7f96f0071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:42.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.927+0000 7f96f6cc2640 1 -- 172.21.15.70:0/843804749 >> v1:172.21.15.70:6789/0 conn(0x7f96f0071550 legacy=0x7f96f012b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:42.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.927+0000 7f96f6cc2640 1 -- 172.21.15.70:0/843804749 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f0071930 con 0x7f96f012b870 2024-09-13T15:43:42.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.928+0000 7f96f6cc2640 1 --2- 172.21.15.70:0/843804749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f012b870 0x7f96f012bd00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009e50 tx=0x7f96e802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170680477b860470 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:42.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.928+0000 7f96f54bf640 1 -- 172.21.15.70:0/843804749 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8031600 con 0x7f96f012b870 2024-09-13T15:43:42.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.928+0000 7f96f54bf640 1 -- 172.21.15.70:0/843804749 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e802d050 con 0x7f96f012b870 2024-09-13T15:43:42.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.929+0000 7f96f54bf640 1 -- 172.21.15.70:0/843804749 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8031980 con 0x7f96f012b870 2024-09-13T15:43:42.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.929+0000 7f96f8725640 1 -- 172.21.15.70:0/843804749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f012b870 msgr2=0x7f96f012bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:42.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.929+0000 7f96f8725640 1 --2- 172.21.15.70:0/843804749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f012b870 0x7f96f012bd00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009e50 tx=0x7f96e802a5f0 comp rx=0 tx=0).stop 2024-09-13T15:43:42.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.929+0000 7f96f8725640 1 -- 172.21.15.70:0/843804749 shutdown_connections 2024-09-13T15:43:42.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.929+0000 7f96f8725640 1 --2- 172.21.15.70:0/843804749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f012b870 0x7f96f012bd00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:42.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.929+0000 7f96f8725640 1 --2- 172.21.15.70:0/843804749 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f96f0070c30 0x7f96f0071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:42.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.929+0000 7f96f8725640 1 -- 172.21.15.70:0/843804749 >> 172.21.15.70:0/843804749 conn(0x7f96f009bc00 msgr2=0x7f96f009c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.929+0000 7f96f8725640 1 -- 172.21.15.70:0/843804749 shutdown_connections 2024-09-13T15:43:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.930+0000 7f96f8725640 1 -- 172.21.15.70:0/843804749 wait complete. 2024-09-13T15:43:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.930+0000 7f96f8725640 1 Processor -- start 2024-09-13T15:43:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.931+0000 7f96f8725640 1 -- start start 2024-09-13T15:43:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.931+0000 7f96f8725640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f96f0070c30 0x7f96f0131ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.931+0000 7f96f8725640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96f0071550 0x7f96f0132020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.931+0000 7f96f8725640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f012b870 0x7f96f0132560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.931+0000 7f96f8725640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f006ce50 con 0x7f96f012b870 2024-09-13T15:43:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.931+0000 7f96f8725640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f008b100 con 0x7f96f0071550 2024-09-13T15:43:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.931+0000 7f96f8725640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96f0132aa0 con 0x7f96f0070c30 2024-09-13T15:43:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.931+0000 7f96f6cc2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f012b870 0x7f96f0132560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.931+0000 7f96f5cc0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96f0071550 0x7f96f0132020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.931+0000 7f96f64c1640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f96f0070c30 0x7f96f0131ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.931+0000 7f96f6cc2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f012b870 0x7f96f0132560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37392/0 (socket says 172.21.15.70:37392) 2024-09-13T15:43:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.931+0000 7f96f6cc2640 1 -- 172.21.15.70:0/1464134852 learned_addr learned my addr 172.21.15.70:0/1464134852 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.932+0000 7f96f64c1640 1 -- 172.21.15.70:0/1464134852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96f0071550 msgr2=0x7f96f0132020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.932+0000 7f96f64c1640 1 --2- 172.21.15.70:0/1464134852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96f0071550 0x7f96f0132020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.932+0000 7f96f64c1640 1 -- 172.21.15.70:0/1464134852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f012b870 msgr2=0x7f96f0132560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.932+0000 7f96f64c1640 1 --2- 172.21.15.70:0/1464134852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f012b870 0x7f96f0132560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.932+0000 7f96f64c1640 1 -- 172.21.15.70:0/1464134852 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e8009b30 con 0x7f96f0070c30 2024-09-13T15:43:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.932+0000 7f96f6cc2640 1 --2- 172.21.15.70:0/1464134852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f012b870 0x7f96f0132560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.932+0000 7f96f64c1640 1 --2- 172.21.15.70:0/1464134852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f96f0070c30 0x7f96f0131ae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00e850 tx=0x7f96ec00ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.932+0000 7f96e6ffd640 1 -- 172.21.15.70:0/1464134852 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0095b0 con 0x7f96f0070c30 2024-09-13T15:43:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.932+0000 7f96e6ffd640 1 -- 172.21.15.70:0/1464134852 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec00c940 con 0x7f96f0070c30 2024-09-13T15:43:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.932+0000 7f96f8725640 1 -- 172.21.15.70:0/1464134852 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f006d780 con 0x7f96f0070c30 2024-09-13T15:43:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.932+0000 7f96e6ffd640 1 -- 172.21.15.70:0/1464134852 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec00ccd0 con 0x7f96f0070c30 2024-09-13T15:43:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.932+0000 7f96f8725640 1 -- 172.21.15.70:0/1464134852 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f01c8970 con 0x7f96f0070c30 2024-09-13T15:43:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.934+0000 7f96f8725640 1 -- 172.21.15.70:0/1464134852 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f0097f00 con 0x7f96f0070c30 2024-09-13T15:43:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.934+0000 7f96e6ffd640 1 -- 172.21.15.70:0/1464134852 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 8) v1 ==== 55050+0+0 (secure 0 0 0) 0x7f96ec013470 con 0x7f96f0070c30 2024-09-13T15:43:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.934+0000 7f96e6ffd640 1 -- 172.21.15.70:0/1464134852 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f96ec05c880 con 0x7f96f0070c30 2024-09-13T15:43:42.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.939+0000 7f96e6ffd640 1 -- 172.21.15.70:0/1464134852 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f96ec03a610 con 0x7f96f0070c30 2024-09-13T15:43:42.970 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.969+0000 7f04786fe640 1 Processor -- start 2024-09-13T15:43:42.970 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.969+0000 7f04786fe640 1 -- start start 2024-09-13T15:43:42.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.969+0000 7f04786fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f047006cab0 0x7f0470136d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:42.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.970+0000 7f04786fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0470137240 0x7f0470139630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:42.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.970+0000 7f0475c99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f047006cab0 0x7f0470136d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:42.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.970+0000 7f047649a640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f047006bfc0 0x7f047006c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58194/0 (socket says 172.21.15.70:58194) 2024-09-13T15:43:42.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.970+0000 7f047649a640 1 -- 172.21.15.70:0/2843528192 learned_addr learned my addr 172.21.15.70:0/2843528192 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:42.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.970+0000 7f0475c99640 1 --2- 172.21.15.70:0/2843528192 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f047006cab0 0x7f0470136d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:54592/0 (socket says 172.21.15.70:54592) 2024-09-13T15:43:42.977 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.970+0000 7f04786fe640 1 -- 172.21.15.70:0/2843528192 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047006de30 con 0x7f047006bfc0 2024-09-13T15:43:42.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.970+0000 7f04786fe640 1 -- 172.21.15.70:0/2843528192 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f047006d9f0 con 0x7f0470137240 2024-09-13T15:43:42.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.970+0000 7f0476c9b640 1 --2- 172.21.15.70:0/2843528192 >> v2:172.21.15.70:3300/0 conn(0x7f0470137240 0x7f0470139630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:42.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.970+0000 7f04786fe640 1 -- 172.21.15.70:0/2843528192 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0470097e80 con 0x7f047006cab0 2024-09-13T15:43:42.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.970+0000 7f0476c9b640 1 -- 172.21.15.70:0/2843528192 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f047006cab0 msgr2=0x7f0470136d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:42.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.970+0000 7f0476c9b640 1 --2- 172.21.15.70:0/2843528192 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f047006cab0 0x7f0470136d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:42.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.970+0000 7f0476c9b640 1 -- 172.21.15.70:0/2843528192 >> v1:172.21.15.70:6789/0 conn(0x7f047006bfc0 legacy=0x7f047006c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:42.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.970+0000 7f0476c9b640 1 -- 172.21.15.70:0/2843528192 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047006d280 con 0x7f0470137240 2024-09-13T15:43:42.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.970+0000 7f0475498640 1 -- 172.21.15.70:0/2843528192 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025180800 0 0) 0x7f047006de30 con 0x7f047006bfc0 2024-09-13T15:43:42.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.970+0000 7f0475c99640 1 --2- 172.21.15.70:0/2843528192 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f047006cab0 0x7f0470136d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:42.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.971+0000 7f0476c9b640 1 --2- 172.21.15.70:0/2843528192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0470137240 0x7f0470139630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0468009d90 tx=0x7f046802a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191612e18c0b3511 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:42.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.971+0000 7f0475498640 1 -- 172.21.15.70:0/2843528192 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468007740 con 0x7f0470137240 2024-09-13T15:43:42.978 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.971+0000 7f0475498640 1 -- 172.21.15.70:0/2843528192 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046802d050 con 0x7f0470137240 2024-09-13T15:43:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.971+0000 7f0475498640 1 -- 172.21.15.70:0/2843528192 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468007ac0 con 0x7f0470137240 2024-09-13T15:43:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.972+0000 7f04786fe640 1 -- 172.21.15.70:0/2843528192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0470137240 msgr2=0x7f0470139630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.972+0000 7f04786fe640 1 --2- 172.21.15.70:0/2843528192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0470137240 0x7f0470139630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0468009d90 tx=0x7f046802a530 comp rx=0 tx=0).stop 2024-09-13T15:43:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.972+0000 7f04786fe640 1 -- 172.21.15.70:0/2843528192 shutdown_connections 2024-09-13T15:43:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.972+0000 7f04786fe640 1 --2- 172.21.15.70:0/2843528192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0470137240 0x7f0470139630 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.972+0000 7f04786fe640 1 --2- 172.21.15.70:0/2843528192 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f047006cab0 0x7f0470136d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.972+0000 7f04786fe640 1 -- 172.21.15.70:0/2843528192 >> 172.21.15.70:0/2843528192 conn(0x7f047009bc00 msgr2=0x7f047009c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.972+0000 7f04786fe640 1 -- 172.21.15.70:0/2843528192 shutdown_connections 2024-09-13T15:43:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.973+0000 7f04786fe640 1 -- 172.21.15.70:0/2843528192 wait complete. 2024-09-13T15:43:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.973+0000 7f04786fe640 1 Processor -- start 2024-09-13T15:43:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.974+0000 7f04786fe640 1 -- start start 2024-09-13T15:43:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.974+0000 7f04786fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f047006bfc0 0x7f04701246e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:42.979 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.974+0000 7f04786fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f047006cab0 0x7f0470124c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.974+0000 7f04786fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0470137240 0x7f0470125160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.974+0000 7f04786fe640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0470125b00 con 0x7f047006bfc0 2024-09-13T15:43:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.974+0000 7f04786fe640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0470126b00 con 0x7f0470137240 2024-09-13T15:43:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.974+0000 7f04786fe640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0470127f00 con 0x7f047006cab0 2024-09-13T15:43:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.974+0000 7f0476c9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0470137240 0x7f0470125160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.974+0000 7f0476c9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0470137240 0x7f0470125160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:54602/0 (socket says 172.21.15.70:54602) 2024-09-13T15:43:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.974+0000 7f0476c9b640 1 -- 172.21.15.70:0/853803109 learned_addr learned my addr 172.21.15.70:0/853803109 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.974+0000 7f0475c99640 1 --2- 172.21.15.70:0/853803109 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f047006cab0 0x7f0470124c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.975+0000 7f0476c9b640 1 -- 172.21.15.70:0/853803109 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f047006cab0 msgr2=0x7f0470124c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.975+0000 7f0476c9b640 1 --2- 172.21.15.70:0/853803109 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f047006cab0 0x7f0470124c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.975+0000 7f0476c9b640 1 -- 172.21.15.70:0/853803109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f047006bfc0 msgr2=0x7f04701246e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T15:43:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.975+0000 7f0476c9b640 1 --2- 172.21.15.70:0/853803109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f047006bfc0 0x7f04701246e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.975+0000 7f0476c9b640 1 -- 172.21.15.70:0/853803109 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0460009400 con 0x7f0470137240 2024-09-13T15:43:42.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.975+0000 7f0475c99640 1 --2- 172.21.15.70:0/853803109 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f047006cab0 0x7f0470124c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:43:42.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.975+0000 7f0476c9b640 1 --2- 172.21.15.70:0/853803109 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0470137240 0x7f0470125160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0468009d90 tx=0x7f0468004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:42.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.975+0000 7f0466ffd640 1 -- 172.21.15.70:0/853803109 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468003b90 con 0x7f0470137240 2024-09-13T15:43:42.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.975+0000 7f0466ffd640 1 -- 172.21.15.70:0/853803109 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0468003cf0 con 0x7f0470137240 2024-09-13T15:43:42.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.975+0000 7f04786fe640 1 -- 172.21.15.70:0/853803109 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0468009a70 con 0x7f0470137240 2024-09-13T15:43:42.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.976+0000 7f0466ffd640 1 -- 172.21.15.70:0/853803109 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046803c930 con 0x7f0470137240 2024-09-13T15:43:42.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.976+0000 7f04786fe640 1 -- 172.21.15.70:0/853803109 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0470136d40 con 0x7f0470137240 2024-09-13T15:43:42.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.976+0000 7f0466ffd640 1 -- 172.21.15.70:0/853803109 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 8) v1 ==== 55050+0+0 (secure 0 0 0) 0x7f046802ad50 con 0x7f0470137240 2024-09-13T15:43:42.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.977+0000 7f0466ffd640 1 -- 172.21.15.70:0/853803109 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f0468082390 con 0x7f0470137240 2024-09-13T15:43:42.982 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.977+0000 7f04786fe640 1 -- 172.21.15.70:0/853803109 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0470071930 con 0x7f0470137240 2024-09-13T15:43:42.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:42.985+0000 7f0466ffd640 1 -- 172.21.15.70:0/853803109 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f0470071930 con 0x7f0470137240 2024-09-13T15:43:43.014 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.013+0000 7f4dd9ee5640 1 Processor -- start 2024-09-13T15:43:43.014 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.013+0000 7f4dd9ee5640 1 -- start start 2024-09-13T15:43:43.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.013+0000 7f4dd9ee5640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4dd412d330 0x7f4dd412d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:43.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.013+0000 7f4dd9ee5640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4dd412a540 0x7f4dd4135670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:43.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.013+0000 7f4dd9ee5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd406d5d0 con 0x7f4dd412dd80 2024-09-13T15:43:43.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.014+0000 7f4dd3fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4dd412a540 0x7f4dd4135670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:43.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.014+0000 7f4dd3fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4dd412a540 0x7f4dd4135670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:54614/0 (socket says 172.21.15.70:54614) 2024-09-13T15:43:43.015 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.014+0000 7f4dd3fff640 1 -- 172.21.15.70:0/963063574 learned_addr learned my addr 172.21.15.70:0/963063574 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:43.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.014+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/963063574 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40971b0 con 0x7f4dd412a540 2024-09-13T15:43:43.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.014+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/963063574 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40979f0 con 0x7f4dd412d330 2024-09-13T15:43:43.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.014+0000 7f4dd37fe640 1 --2- 172.21.15.70:0/963063574 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4dd412d330 0x7f4dd412d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:43.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.014+0000 7f4dd3fff640 1 -- 172.21.15.70:0/963063574 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4dd412d330 msgr2=0x7f4dd412d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:43.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.014+0000 7f4dd3fff640 1 --2- 172.21.15.70:0/963063574 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4dd412d330 0x7f4dd412d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.014+0000 7f4dd3fff640 1 -- 172.21.15.70:0/963063574 >> v1:172.21.15.70:6789/0 conn(0x7f4dd412dd80 legacy=0x7f4dd4129e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.016 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.014+0000 7f4dd27fc640 1 -- 172.21.15.70:0/963063574 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988916330 0 0) 0x7f4dd406d5d0 con 0x7f4dd412dd80 2024-09-13T15:43:43.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.014+0000 7f4dd3fff640 1 -- 172.21.15.70:0/963063574 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd412e1a0 con 0x7f4dd412a540 2024-09-13T15:43:43.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.015+0000 7f4dd3fff640 1 --2- 172.21.15.70:0/963063574 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4dd412a540 0x7f4dd4135670 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009e80 tx=0x7f4dcc02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d00dab9522c43df8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:43.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.015+0000 7f4dd27fc640 1 -- 172.21.15.70:0/963063574 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc031400 con 0x7f4dd412a540 2024-09-13T15:43:43.017 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.015+0000 7f4dd27fc640 1 -- 172.21.15.70:0/963063574 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc02d050 con 0x7f4dd412a540 2024-09-13T15:43:43.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.015+0000 7f4dd27fc640 1 -- 172.21.15.70:0/963063574 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc031780 con 0x7f4dd412a540 2024-09-13T15:43:43.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.016+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/963063574 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4dd412a540 msgr2=0x7f4dd4135670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.016+0000 7f4dd9ee5640 1 --2- 172.21.15.70:0/963063574 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4dd412a540 0x7f4dd4135670 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009e80 tx=0x7f4dcc02a5d0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.016+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/963063574 shutdown_connections 2024-09-13T15:43:43.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.016+0000 7f4dd9ee5640 1 --2- 172.21.15.70:0/963063574 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4dd412a540 0x7f4dd4135670 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.016+0000 7f4dd9ee5640 1 --2- 172.21.15.70:0/963063574 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4dd412d330 0x7f4dd412d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.016+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/963063574 >> 172.21.15.70:0/963063574 conn(0x7f4dd40a0060 msgr2=0x7f4dd40a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:43.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.016+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/963063574 shutdown_connections 2024-09-13T15:43:43.018 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.016+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/963063574 wait complete. 2024-09-13T15:43:43.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.017+0000 7f4dd9ee5640 1 Processor -- start 2024-09-13T15:43:43.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.017+0000 7f4dd9ee5640 1 -- start start 2024-09-13T15:43:43.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd9ee5640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4dd412a540 0x7f4dd412e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:43.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd9ee5640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4dd412d330 0x7f4dd412eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:43.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd9ee5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4dd412dd80 0x7f4dd412f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:43.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd9ee5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd4131570 con 0x7f4dd412dd80 2024-09-13T15:43:43.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd9ee5640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd4132120 con 0x7f4dd412a540 2024-09-13T15:43:43.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd9ee5640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd4130d00 con 0x7f4dd412d330 2024-09-13T15:43:43.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd37fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4dd412a540 0x7f4dd412e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:43.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd37fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4dd412a540 0x7f4dd412e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:54626/0 (socket says 172.21.15.70:54626) 2024-09-13T15:43:43.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd37fe640 1 -- 172.21.15.70:0/3468958285 learned_addr learned my addr 172.21.15.70:0/3468958285 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:43.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd2ffd640 1 --2- 172.21.15.70:0/3468958285 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4dd412d330 0x7f4dd412eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:43.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd3fff640 1 --2- 172.21.15.70:0/3468958285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4dd412dd80 0x7f4dd412f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:43.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd2ffd640 1 -- 172.21.15.70:0/3468958285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4dd412a540 msgr2=0x7f4dd412e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd2ffd640 1 --2- 172.21.15.70:0/3468958285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4dd412a540 0x7f4dd412e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd2ffd640 1 -- 172.21.15.70:0/3468958285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4dd412dd80 msgr2=0x7f4dd412f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd2ffd640 1 --2- 172.21.15.70:0/3468958285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4dd412dd80 0x7f4dd412f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd2ffd640 1 -- 172.21.15.70:0/3468958285 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc009b30 con 0x7f4dd412d330 2024-09-13T15:43:43.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.018+0000 7f4dd3fff640 1 --2- 172.21.15.70:0/3468958285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4dd412dd80 0x7f4dd412f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:43:43.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.019+0000 7f4dd2ffd640 1 --2- 172.21.15.70:0/3468958285 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4dd412d330 0x7f4dd412eec0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800c450 tx=0x7f4dc800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:43.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.019+0000 7f4dbbfff640 1 -- 172.21.15.70:0/3468958285 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc80035f0 con 0x7f4dd412d330 2024-09-13T15:43:43.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.019+0000 7f4dbbfff640 1 -- 172.21.15.70:0/3468958285 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc8003750 con 0x7f4dd412d330 2024-09-13T15:43:43.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.019+0000 7f4dbbfff640 1 -- 172.21.15.70:0/3468958285 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8003b00 con 0x7f4dd412d330 2024-09-13T15:43:43.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.019+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/3468958285 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd4130420 con 0x7f4dd412d330 2024-09-13T15:43:43.022 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.019+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/3468958285 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd41352f0 con 0x7f4dd412d330 2024-09-13T15:43:43.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.020+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/3468958285 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd412d840 con 0x7f4dd412d330 2024-09-13T15:43:43.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.020+0000 7f4dbbfff640 1 -- 172.21.15.70:0/3468958285 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 8) v1 ==== 55050+0+0 (secure 0 0 0) 0x7f4dc8003c60 con 0x7f4dd412d330 2024-09-13T15:43:43.023 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.021+0000 7f4dbbfff640 1 -- 172.21.15.70:0/3468958285 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f4dc805bc10 con 0x7f4dd412d330 2024-09-13T15:43:43.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.024+0000 7f4dbbfff640 1 -- 172.21.15.70:0/3468958285 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f4dc8039a00 con 0x7f4dd412d330 2024-09-13T15:43:43.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.033+0000 7f592fdbc640 1 Processor -- start 2024-09-13T15:43:43.034 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.033+0000 7f592fdbc640 1 -- start start 2024-09-13T15:43:43.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.034+0000 7f592fdbc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f592812d330 0x7f592812d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:43.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.034+0000 7f592fdbc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f592812dd80 0x7f5928129e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:43.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.034+0000 7f592fdbc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592806d5d0 con 0x7f592812a370 2024-09-13T15:43:43.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.034+0000 7f592d357640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f592812dd80 0x7f5928129e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:43.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.034+0000 7f592db58640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f592812d330 0x7f592812d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:43.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.034+0000 7f592d357640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f592812dd80 0x7f5928129e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:54638/0 (socket says 172.21.15.70:54638) 2024-09-13T15:43:43.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.034+0000 7f592d357640 1 -- 172.21.15.70:0/2267449669 learned_addr learned my addr 172.21.15.70:0/2267449669 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:43.036 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.034+0000 7f592fdbc640 1 -- 172.21.15.70:0/2267449669 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59280971b0 con 0x7f592812dd80 2024-09-13T15:43:43.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.034+0000 7f592fdbc640 1 -- 172.21.15.70:0/2267449669 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59280979f0 con 0x7f592812d330 2024-09-13T15:43:43.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.035+0000 7f592cb56640 1 -- 172.21.15.70:0/2267449669 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029927053 0 0) 0x7f592806d5d0 con 0x7f592812a370 2024-09-13T15:43:43.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.035+0000 7f592cb56640 1 -- 172.21.15.70:0/2267449669 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5914003520 con 0x7f592812a370 2024-09-13T15:43:43.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.035+0000 7f592d357640 1 -- 172.21.15.70:0/2267449669 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f592812d330 msgr2=0x7f592812d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:43.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.035+0000 7f592d357640 1 --2- 172.21.15.70:0/2267449669 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f592812d330 0x7f592812d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.035+0000 7f592d357640 1 -- 172.21.15.70:0/2267449669 >> v1:172.21.15.70:6789/0 conn(0x7f592812a370 legacy=0x7f5928134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.037 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.035+0000 7f592d357640 1 -- 172.21.15.70:0/2267449669 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592812e1a0 con 0x7f592812dd80 2024-09-13T15:43:43.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.036+0000 7f592d357640 1 --2- 172.21.15.70:0/2267449669 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f592812dd80 0x7f5928129e30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f59180098a0 tx=0x7f591802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d1b4dc522b14d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:43.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.036+0000 7f592cb56640 1 -- 172.21.15.70:0/2267449669 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591802ae60 con 0x7f592812dd80 2024-09-13T15:43:43.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.036+0000 7f592cb56640 1 -- 172.21.15.70:0/2267449669 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5918032440 con 0x7f592812dd80 2024-09-13T15:43:43.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.036+0000 7f592cb56640 1 -- 172.21.15.70:0/2267449669 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59180327b0 con 0x7f592812dd80 2024-09-13T15:43:43.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.036+0000 7f592fdbc640 1 -- 172.21.15.70:0/2267449669 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f592812dd80 msgr2=0x7f5928129e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.036+0000 7f592fdbc640 1 --2- 172.21.15.70:0/2267449669 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f592812dd80 0x7f5928129e30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f59180098a0 tx=0x7f591802a110 comp rx=0 tx=0).stop 2024-09-13T15:43:43.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.037+0000 7f592fdbc640 1 -- 172.21.15.70:0/2267449669 shutdown_connections 2024-09-13T15:43:43.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.037+0000 7f592fdbc640 1 --2- 172.21.15.70:0/2267449669 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f592812dd80 0x7f5928129e30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.037+0000 7f592fdbc640 1 --2- 172.21.15.70:0/2267449669 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f592812d330 0x7f592812d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.037+0000 7f592fdbc640 1 -- 172.21.15.70:0/2267449669 >> 172.21.15.70:0/2267449669 conn(0x7f59280a0060 msgr2=0x7f59280a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:43.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.037+0000 7f592fdbc640 1 -- 172.21.15.70:0/2267449669 shutdown_connections 2024-09-13T15:43:43.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.037+0000 7f592fdbc640 1 -- 172.21.15.70:0/2267449669 wait complete. 2024-09-13T15:43:43.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.038+0000 7f592fdbc640 1 Processor -- start 2024-09-13T15:43:43.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.038+0000 7f592fdbc640 1 -- start start 2024-09-13T15:43:43.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592fdbc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f592812a370 0x7f592808bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:43.041 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592fdbc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f592812d330 0x7f592808c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592fdbc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f592812dd80 0x7f592808ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592fdbc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592808f780 con 0x7f592812dd80 2024-09-13T15:43:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592fdbc640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592808f320 con 0x7f592812a370 2024-09-13T15:43:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592db58640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f592812a370 0x7f592808bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592fdbc640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f592808e2d0 con 0x7f592812d330 2024-09-13T15:43:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592d357640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f592812d330 0x7f592808c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592d357640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f592812d330 0x7f592808c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:38184/0 (socket says 172.21.15.70:38184) 2024-09-13T15:43:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592d357640 1 -- 172.21.15.70:0/2436226329 learned_addr learned my addr 172.21.15.70:0/2436226329 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592e359640 1 --2- 172.21.15.70:0/2436226329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f592812dd80 0x7f592808ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592d357640 1 -- 172.21.15.70:0/2436226329 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f592812a370 msgr2=0x7f592808bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592d357640 1 --2- 172.21.15.70:0/2436226329 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f592812a370 0x7f592808bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592d357640 1 -- 172.21.15.70:0/2436226329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f592812dd80 msgr2=0x7f592808ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592d357640 1 --2- 172.21.15.70:0/2436226329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f592812dd80 0x7f592808ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592d357640 1 -- 172.21.15.70:0/2436226329 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59180094d0 con 0x7f592812d330 2024-09-13T15:43:43.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.039+0000 7f592db58640 1 --2- 172.21.15.70:0/2436226329 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f592812a370 0x7f592808bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:43:43.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.040+0000 7f592d357640 1 --2- 172.21.15.70:0/2436226329 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f592812d330 0x7f592808c4d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5918035040 tx=0x7f59180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:43.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.040+0000 7f591e7fc640 1 -- 172.21.15.70:0/2436226329 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918003e90 con 0x7f592812d330 2024-09-13T15:43:43.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.040+0000 7f592fdbc640 1 -- 172.21.15.70:0/2436226329 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592808d9c0 con 0x7f592812d330 2024-09-13T15:43:43.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.040+0000 7f591e7fc640 1 -- 172.21.15.70:0/2436226329 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59180325b0 con 0x7f592812d330 2024-09-13T15:43:43.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.040+0000 7f591e7fc640 1 -- 172.21.15.70:0/2436226329 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918032920 con 0x7f592812d330 2024-09-13T15:43:43.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.040+0000 7f592fdbc640 1 -- 172.21.15.70:0/2436226329 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59281c2390 con 0x7f592812d330 2024-09-13T15:43:43.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.041+0000 7f591e7fc640 1 -- 172.21.15.70:0/2436226329 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 8) v1 ==== 55050+0+0 (secure 0 0 0) 0x7f5918039040 con 0x7f592812d330 2024-09-13T15:43:43.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.041+0000 7f592fdbc640 1 -- 172.21.15.70:0/2436226329 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592812d840 con 0x7f592812d330 2024-09-13T15:43:43.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.042+0000 7f591e7fc640 1 -- 172.21.15.70:0/2436226329 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f591807ccd0 con 0x7f592812d330 2024-09-13T15:43:43.050 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.049+0000 7f591e7fc640 1 -- 172.21.15.70:0/2436226329 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f591805aff0 con 0x7f592812d330 2024-09-13T15:43:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.090+0000 7f96f8725640 1 -- 172.21.15.70:0/1464134852 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f96f0071010 con 0x7f96f0070c30 2024-09-13T15:43:43.092 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.090+0000 7f96e6ffd640 1 -- 172.21.15.70:0/1464134852 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f96ec021070 con 0x7f96f0070c30 2024-09-13T15:43:43.092 INFO:teuthology.orchestra.run.smithi070.stdout:38654705681 2024-09-13T15:43:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.092+0000 7f96f8725640 1 -- 172.21.15.70:0/1464134852 >> v1:172.21.15.70:6816/1229510400 conn(0x7f96d40473e0 legacy=0x7f96d40498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.092+0000 7f96f8725640 1 -- 172.21.15.70:0/1464134852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f96f0070c30 msgr2=0x7f96f0131ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.092+0000 7f96f8725640 1 --2- 172.21.15.70:0/1464134852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f96f0070c30 0x7f96f0131ae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00e850 tx=0x7f96ec00ed10 comp rx=0 tx=0).stop 2024-09-13T15:43:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.092+0000 7f96f8725640 1 -- 172.21.15.70:0/1464134852 shutdown_connections 2024-09-13T15:43:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.092+0000 7f96f8725640 1 --2- 172.21.15.70:0/1464134852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96f012b870 0x7f96f0132560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.092+0000 7f96f8725640 1 --2- 172.21.15.70:0/1464134852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f96f0071550 0x7f96f0132020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.092+0000 7f96f8725640 1 --2- 172.21.15.70:0/1464134852 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f96f0070c30 0x7f96f0131ae0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.092+0000 7f96f8725640 1 -- 172.21.15.70:0/1464134852 >> 172.21.15.70:0/1464134852 conn(0x7f96f009bc00 msgr2=0x7f96f0124bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.092+0000 7f96f8725640 1 -- 172.21.15.70:0/1464134852 shutdown_connections 2024-09-13T15:43:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.092+0000 7f96f8725640 1 -- 172.21.15.70:0/1464134852 wait complete. 2024-09-13T15:43:43.104 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705681 for osd.2 2024-09-13T15:43:43.133 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.132+0000 7f04786fe640 1 -- 172.21.15.70:0/853803109 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0470071930 con 0x7f0470137240 2024-09-13T15:43:43.134 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.132+0000 7f0466ffd640 1 -- 172.21.15.70:0/853803109 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0470071930 con 0x7f0470137240 2024-09-13T15:43:43.134 INFO:teuthology.orchestra.run.smithi070.stdout:38654705681 2024-09-13T15:43:43.135 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.134+0000 7f04786fe640 1 -- 172.21.15.70:0/853803109 >> v1:172.21.15.70:6816/1229510400 conn(0x7f04540473b0 legacy=0x7f0454049870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.134+0000 7f04786fe640 1 -- 172.21.15.70:0/853803109 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0470137240 msgr2=0x7f0470125160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.134+0000 7f04786fe640 1 --2- 172.21.15.70:0/853803109 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0470137240 0x7f0470125160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0468009d90 tx=0x7f0468004a40 comp rx=0 tx=0).stop 2024-09-13T15:43:43.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.134+0000 7f04786fe640 1 -- 172.21.15.70:0/853803109 shutdown_connections 2024-09-13T15:43:43.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.134+0000 7f04786fe640 1 --2- 172.21.15.70:0/853803109 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0470137240 0x7f0470125160 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.134+0000 7f04786fe640 1 --2- 172.21.15.70:0/853803109 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f047006cab0 0x7f0470124c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.134+0000 7f04786fe640 1 --2- 172.21.15.70:0/853803109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f047006bfc0 0x7f04701246e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.134+0000 7f04786fe640 1 -- 172.21.15.70:0/853803109 >> 172.21.15.70:0/853803109 conn(0x7f047009bc00 msgr2=0x7f0470098fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:43.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.134+0000 7f04786fe640 1 -- 172.21.15.70:0/853803109 shutdown_connections 2024-09-13T15:43:43.136 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.134+0000 7f04786fe640 1 -- 172.21.15.70:0/853803109 wait complete. 2024-09-13T15:43:43.146 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705681 for osd.3 2024-09-13T15:43:43.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.169+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/3468958285 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4dd4126410 con 0x7f4dd412d330 2024-09-13T15:43:43.171 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.170+0000 7f4dbbfff640 1 -- 172.21.15.70:0/3468958285 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4dc8017020 con 0x7f4dd412d330 2024-09-13T15:43:43.171 INFO:teuthology.orchestra.run.smithi070.stdout:38654705681 2024-09-13T15:43:43.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.171+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/3468958285 >> v1:172.21.15.70:6816/1229510400 conn(0x7f4dc00473e0 legacy=0x7f4dc00498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.171+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/3468958285 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4dd412d330 msgr2=0x7f4dd412eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.171+0000 7f4dd9ee5640 1 --2- 172.21.15.70:0/3468958285 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4dd412d330 0x7f4dd412eec0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800c450 tx=0x7f4dc800c910 comp rx=0 tx=0).stop 2024-09-13T15:43:43.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.172+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/3468958285 shutdown_connections 2024-09-13T15:43:43.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.172+0000 7f4dd9ee5640 1 --2- 172.21.15.70:0/3468958285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4dd412dd80 0x7f4dd412f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.172+0000 7f4dd9ee5640 1 --2- 172.21.15.70:0/3468958285 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4dd412d330 0x7f4dd412eec0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.172+0000 7f4dd9ee5640 1 --2- 172.21.15.70:0/3468958285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4dd412a540 0x7f4dd412e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.173 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.172+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/3468958285 >> 172.21.15.70:0/3468958285 conn(0x7f4dd40a0060 msgr2=0x7f4dd40984d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:43.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.172+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/3468958285 shutdown_connections 2024-09-13T15:43:43.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.172+0000 7f4dd9ee5640 1 -- 172.21.15.70:0/3468958285 wait complete. 2024-09-13T15:43:43.183 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705681 for osd.0 2024-09-13T15:43:43.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.193+0000 7f592fdbc640 1 -- 172.21.15.70:0/2436226329 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f592806a6b0 con 0x7f592812d330 2024-09-13T15:43:43.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.193+0000 7f591e7fc640 1 -- 172.21.15.70:0/2436226329 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5918037090 con 0x7f592812d330 2024-09-13T15:43:43.195 INFO:teuthology.orchestra.run.smithi070.stdout:38654705680 2024-09-13T15:43:43.196 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.195+0000 7f592fdbc640 1 -- 172.21.15.70:0/2436226329 >> v1:172.21.15.70:6816/1229510400 conn(0x7f590c0473b0 legacy=0x7f590c049870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.195+0000 7f592fdbc640 1 -- 172.21.15.70:0/2436226329 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f592812d330 msgr2=0x7f592808c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:43.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.195+0000 7f592fdbc640 1 --2- 172.21.15.70:0/2436226329 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f592812d330 0x7f592808c4d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5918035040 tx=0x7f59180046c0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.195+0000 7f592fdbc640 1 -- 172.21.15.70:0/2436226329 shutdown_connections 2024-09-13T15:43:43.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.195+0000 7f592fdbc640 1 --2- 172.21.15.70:0/2436226329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f592812dd80 0x7f592808ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.195+0000 7f592fdbc640 1 --2- 172.21.15.70:0/2436226329 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f592812d330 0x7f592808c4d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.195+0000 7f592fdbc640 1 --2- 172.21.15.70:0/2436226329 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f592812a370 0x7f592808bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:43.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.195+0000 7f592fdbc640 1 -- 172.21.15.70:0/2436226329 >> 172.21.15.70:0/2436226329 conn(0x7f59280a0060 msgr2=0x7f5928097cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:43.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.195+0000 7f592fdbc640 1 -- 172.21.15.70:0/2436226329 shutdown_connections 2024-09-13T15:43:43.197 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:43.195+0000 7f592fdbc640 1 -- 172.21.15.70:0/2436226329 wait complete. 2024-09-13T15:43:43.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705681 got 38654705680 for osd.1 2024-09-13T15:43:44.106 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T15:43:44.146 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T15:43:44.184 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T15:43:44.207 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T15:43:44.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.209+0000 7f3195a41640 1 Processor -- start 2024-09-13T15:43:44.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.209+0000 7f3195a41640 1 -- start start 2024-09-13T15:43:44.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.210+0000 7f3195a41640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f319006cdd0 0x7f319006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.210+0000 7f3195a41640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f319012a810 0x7f319012ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.210+0000 7f3195a41640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f319006f350 con 0x7f319006d6f0 2024-09-13T15:43:44.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.210+0000 7f3195a41640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f319006e930 con 0x7f319012a810 2024-09-13T15:43:44.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.210+0000 7f3195a41640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319006daf0 con 0x7f319006cdd0 2024-09-13T15:43:44.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.210+0000 7f318effd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f319006cdd0 0x7f319006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.212 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.210+0000 7f318f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f319012a810 0x7f319012ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.210+0000 7f318effd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f319006cdd0 0x7f319006d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:54684/0 (socket says 172.21.15.70:54684) 2024-09-13T15:43:44.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.210+0000 7f318effd640 1 -- 172.21.15.70:0/3756579370 learned_addr learned my addr 172.21.15.70:0/3756579370 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:44.213 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.210+0000 7f318e7fc640 1 --1- 172.21.15.70:0/3756579370 >> v1:172.21.15.70:6789/0 conn(0x7f319006d6f0 0x7f319012a100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58228/0 (socket says 172.21.15.70:58228) 2024-09-13T15:43:44.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.210+0000 7f318f7fe640 1 -- 172.21.15.70:0/3756579370 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f319006cdd0 msgr2=0x7f319006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:44.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.210+0000 7f318f7fe640 1 --2- 172.21.15.70:0/3756579370 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f319006cdd0 0x7f319006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.210+0000 7f318f7fe640 1 -- 172.21.15.70:0/3756579370 >> v1:172.21.15.70:6789/0 conn(0x7f319006d6f0 legacy=0x7f319012a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.210+0000 7f318f7fe640 1 -- 172.21.15.70:0/3756579370 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319006dcf0 con 0x7f319012a810 2024-09-13T15:43:44.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.211+0000 7f318f7fe640 1 --2- 172.21.15.70:0/3756579370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319012a810 0x7f319012ac20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3188009d90 tx=0x7f318802a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd549f2a48199a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:44.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.211+0000 7f318dffb640 1 -- 172.21.15.70:0/3756579370 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188007740 con 0x7f319012a810 2024-09-13T15:43:44.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.211+0000 7f318dffb640 1 -- 172.21.15.70:0/3756579370 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318802d050 con 0x7f319012a810 2024-09-13T15:43:44.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.212+0000 7f318dffb640 1 -- 172.21.15.70:0/3756579370 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188007ac0 con 0x7f319012a810 2024-09-13T15:43:44.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.212+0000 7f3195a41640 1 -- 172.21.15.70:0/3756579370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319012a810 msgr2=0x7f319012ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.212+0000 7f3195a41640 1 --2- 172.21.15.70:0/3756579370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319012a810 0x7f319012ac20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3188009d90 tx=0x7f318802a530 comp rx=0 tx=0).stop 2024-09-13T15:43:44.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.213+0000 7f3195a41640 1 -- 172.21.15.70:0/3756579370 shutdown_connections 2024-09-13T15:43:44.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.213+0000 7f3195a41640 1 --2- 172.21.15.70:0/3756579370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319012a810 0x7f319012ac20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.213+0000 7f3195a41640 1 --2- 172.21.15.70:0/3756579370 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f319006cdd0 0x7f319006d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.213+0000 7f3195a41640 1 -- 172.21.15.70:0/3756579370 >> 172.21.15.70:0/3756579370 conn(0x7f31900973c0 msgr2=0x7f31900977d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:44.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.213+0000 7f3195a41640 1 -- 172.21.15.70:0/3756579370 shutdown_connections 2024-09-13T15:43:44.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.213+0000 7f3195a41640 1 -- 172.21.15.70:0/3756579370 wait complete. 2024-09-13T15:43:44.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.213+0000 7f3195a41640 1 Processor -- start 2024-09-13T15:43:44.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.214+0000 7f3195a41640 1 -- start start 2024-09-13T15:43:44.219 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.214+0000 7f3195a41640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f319006cdd0 0x7f319008aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.214+0000 7f3195a41640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319006d6f0 0x7f319008b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.214+0000 7f3195a41640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f319012a810 0x7f319008ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.214+0000 7f3195a41640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319008e700 con 0x7f319006d6f0 2024-09-13T15:43:44.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.214+0000 7f3195a41640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3190124a40 con 0x7f319006cdd0 2024-09-13T15:43:44.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.214+0000 7f3195a41640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f319008e270 con 0x7f319012a810 2024-09-13T15:43:44.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.214+0000 7f318e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319006d6f0 0x7f319008b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.214+0000 7f318e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319006d6f0 0x7f319008b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37448/0 (socket says 172.21.15.70:37448) 2024-09-13T15:43:44.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.214+0000 7f318e7fc640 1 -- 172.21.15.70:0/2047086579 learned_addr learned my addr 172.21.15.70:0/2047086579 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:44.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.214+0000 7f318effd640 1 --2- 172.21.15.70:0/2047086579 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f319006cdd0 0x7f319008aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.215+0000 7f318f7fe640 1 --2- 172.21.15.70:0/2047086579 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f319012a810 0x7f319008ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.220 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.215+0000 7f318e7fc640 1 -- 172.21.15.70:0/2047086579 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f319012a810 msgr2=0x7f319008ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.215+0000 7f318e7fc640 1 --2- 172.21.15.70:0/2047086579 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f319012a810 0x7f319008ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.215+0000 7f318e7fc640 1 -- 172.21.15.70:0/2047086579 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f319006cdd0 msgr2=0x7f319008aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.215+0000 7f318e7fc640 1 --2- 172.21.15.70:0/2047086579 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f319006cdd0 0x7f319008aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.215+0000 7f318e7fc640 1 -- 172.21.15.70:0/2047086579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3188009a70 con 0x7f319006d6f0 2024-09-13T15:43:44.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.215+0000 7f318effd640 1 --2- 172.21.15.70:0/2047086579 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f319006cdd0 0x7f319008aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:44.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.215+0000 7f318e7fc640 1 --2- 172.21.15.70:0/2047086579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319006d6f0 0x7f319008b530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f318400c4d0 tx=0x7f318400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:44.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.215+0000 7f3177fff640 1 -- 172.21.15.70:0/2047086579 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3184007480 con 0x7f319006d6f0 2024-09-13T15:43:44.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.215+0000 7f3177fff640 1 -- 172.21.15.70:0/2047086579 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31840075e0 con 0x7f319006d6f0 2024-09-13T15:43:44.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.215+0000 7f3195a41640 1 -- 172.21.15.70:0/2047086579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319008ca20 con 0x7f319006d6f0 2024-09-13T15:43:44.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.216+0000 7f3177fff640 1 -- 172.21.15.70:0/2047086579 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31840049e0 con 0x7f319006d6f0 2024-09-13T15:43:44.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.216+0000 7f3195a41640 1 -- 172.21.15.70:0/2047086579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31901be230 con 0x7f319006d6f0 2024-09-13T15:43:44.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.217+0000 7f3177fff640 1 -- 172.21.15.70:0/2047086579 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 9) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f3184004b40 con 0x7f319006d6f0 2024-09-13T15:43:44.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.217+0000 7f3195a41640 1 -- 172.21.15.70:0/2047086579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3190070760 con 0x7f319006d6f0 2024-09-13T15:43:44.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.217+0000 7f3177fff640 1 -- 172.21.15.70:0/2047086579 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f318405be70 con 0x7f319006d6f0 2024-09-13T15:43:44.223 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.222+0000 7f3177fff640 1 -- 172.21.15.70:0/2047086579 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f3184039ac0 con 0x7f319006d6f0 2024-09-13T15:43:44.259 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.258+0000 7f2eb7ae9640 1 Processor -- start 2024-09-13T15:43:44.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.259+0000 7f2eb7ae9640 1 -- start start 2024-09-13T15:43:44.263 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.259+0000 7f2eb7ae9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2eb012d330 0x7f2eb012d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.263 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.259+0000 7f2eb7ae9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f2eb012dd80 0x7f2eb0129e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.259+0000 7f2eb7ae9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2eb012a370 0x7f2eb012a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.259+0000 7f2eb7ae9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb006d5d0 con 0x7f2eb012d330 2024-09-13T15:43:44.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.259+0000 7f2eb7ae9640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb00971b0 con 0x7f2eb012a370 2024-09-13T15:43:44.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.259+0000 7f2eb7ae9640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb00979f0 con 0x7f2eb012dd80 2024-09-13T15:43:44.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.259+0000 7f2eb6086640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2eb012a370 0x7f2eb012a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.259+0000 7f2eb6086640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2eb012a370 0x7f2eb012a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:54710/0 (socket says 172.21.15.70:54710) 2024-09-13T15:43:44.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.259+0000 7f2eb6086640 1 -- 172.21.15.70:0/3246701876 learned_addr learned my addr 172.21.15.70:0/3246701876 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:44.264 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.259+0000 7f2eb5084640 1 --2- 172.21.15.70:0/3246701876 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f2eb012dd80 0x7f2eb0129e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.260+0000 7f2eb5885640 1 --2- 172.21.15.70:0/3246701876 >> v2:172.21.15.70:3300/0 conn(0x7f2eb012d330 0x7f2eb012d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.260+0000 7f2eb6086640 1 -- 172.21.15.70:0/3246701876 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f2eb012dd80 msgr2=0x7f2eb0129e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:44.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.260+0000 7f2eb6086640 1 --2- 172.21.15.70:0/3246701876 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f2eb012dd80 0x7f2eb0129e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.260+0000 7f2eb6086640 1 -- 172.21.15.70:0/3246701876 >> v2:172.21.15.70:3300/0 conn(0x7f2eb012d330 msgr2=0x7f2eb012d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:44.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.260+0000 7f2eb6086640 1 --2- 172.21.15.70:0/3246701876 >> v2:172.21.15.70:3300/0 conn(0x7f2eb012d330 0x7f2eb012d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.260+0000 7f2eb6086640 1 -- 172.21.15.70:0/3246701876 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb012e1a0 con 0x7f2eb012a370 2024-09-13T15:43:44.265 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.261+0000 7f2eb5885640 1 --2- 172.21.15.70:0/3246701876 >> v2:172.21.15.70:3300/0 conn(0x7f2eb012d330 0x7f2eb012d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:43:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.261+0000 7f2eb6086640 1 --2- 172.21.15.70:0/3246701876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2eb012a370 0x7f2eb012a800 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009e80 tx=0x7f2ea802a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa36e6826c08789 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.261+0000 7f2eb4883640 1 -- 172.21.15.70:0/3246701876 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8031600 con 0x7f2eb012a370 2024-09-13T15:43:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.261+0000 7f2eb4883640 1 -- 172.21.15.70:0/3246701876 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea802d050 con 0x7f2eb012a370 2024-09-13T15:43:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.261+0000 7f2eb4883640 1 -- 172.21.15.70:0/3246701876 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea80319c0 con 0x7f2eb012a370 2024-09-13T15:43:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.262+0000 7f2eb7ae9640 1 -- 172.21.15.70:0/3246701876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2eb012a370 msgr2=0x7f2eb012a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.262+0000 7f2eb7ae9640 1 --2- 172.21.15.70:0/3246701876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2eb012a370 0x7f2eb012a800 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009e80 tx=0x7f2ea802a5b0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.262+0000 7f2eb7ae9640 1 -- 172.21.15.70:0/3246701876 shutdown_connections 2024-09-13T15:43:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.262+0000 7f2eb7ae9640 1 --2- 172.21.15.70:0/3246701876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2eb012a370 0x7f2eb012a800 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.262+0000 7f2eb7ae9640 1 --2- 172.21.15.70:0/3246701876 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f2eb012dd80 0x7f2eb0129e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.262+0000 7f2eb7ae9640 1 --2- 172.21.15.70:0/3246701876 >> v2:172.21.15.70:3300/0 conn(0x7f2eb012d330 0x7f2eb012d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.262+0000 7f2eb7ae9640 1 -- 172.21.15.70:0/3246701876 >> 172.21.15.70:0/3246701876 conn(0x7f2eb00a0060 msgr2=0x7f2eb00a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:44.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.262+0000 7f2eb7ae9640 1 -- 172.21.15.70:0/3246701876 shutdown_connections 2024-09-13T15:43:44.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.262+0000 7f2eb7ae9640 1 -- 172.21.15.70:0/3246701876 wait complete. 2024-09-13T15:43:44.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.263+0000 7f2eb7ae9640 1 Processor -- start 2024-09-13T15:43:44.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.263+0000 7f2eb7ae9640 1 -- start start 2024-09-13T15:43:44.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.263+0000 7f2eb7ae9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2eb012a370 0x7f2eb0132950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.263+0000 7f2eb7ae9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f2eb012d330 0x7f2eb0132e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.263+0000 7f2eb7ae9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb012dd80 0x7f2eb01333d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.263+0000 7f2eb7ae9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb0131490 con 0x7f2eb012dd80 2024-09-13T15:43:44.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.263+0000 7f2eb7ae9640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb0131030 con 0x7f2eb012a370 2024-09-13T15:43:44.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.263+0000 7f2eb7ae9640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb0132420 con 0x7f2eb012d330 2024-09-13T15:43:44.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.263+0000 7f2eb5084640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f2eb012d330 0x7f2eb0132e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.263+0000 7f2eb5084640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f2eb012d330 0x7f2eb0132e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:38232/0 (socket says 172.21.15.70:38232) 2024-09-13T15:43:44.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.263+0000 7f2eb5084640 1 -- 172.21.15.70:0/3907094926 learned_addr learned my addr 172.21.15.70:0/3907094926 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:44.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.263+0000 7f2eb5885640 1 --2- 172.21.15.70:0/3907094926 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2eb012a370 0x7f2eb0132950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.264+0000 7f2eb6086640 1 --2- 172.21.15.70:0/3907094926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb012dd80 0x7f2eb01333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.264+0000 7f2eb5084640 1 -- 172.21.15.70:0/3907094926 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2eb012a370 msgr2=0x7f2eb0132950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.264+0000 7f2eb5084640 1 --2- 172.21.15.70:0/3907094926 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2eb012a370 0x7f2eb0132950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.264+0000 7f2eb5084640 1 -- 172.21.15.70:0/3907094926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb012dd80 msgr2=0x7f2eb01333d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.264+0000 7f2eb5084640 1 --2- 172.21.15.70:0/3907094926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb012dd80 0x7f2eb01333d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.264+0000 7f2eb5084640 1 -- 172.21.15.70:0/3907094926 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea8009b30 con 0x7f2eb012d330 2024-09-13T15:43:44.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.264+0000 7f2eb6086640 1 --2- 172.21.15.70:0/3907094926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb012dd80 0x7f2eb01333d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:43:44.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.264+0000 7f2eb5084640 1 --2- 172.21.15.70:0/3907094926 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f2eb012d330 0x7f2eb0132e90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000e9e0 tx=0x7f2ea000eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:44.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.264+0000 7f2ea67fc640 1 -- 172.21.15.70:0/3907094926 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea000c9d0 con 0x7f2eb012d330 2024-09-13T15:43:44.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.264+0000 7f2eb7ae9640 1 -- 172.21.15.70:0/3907094926 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb0130720 con 0x7f2eb012d330 2024-09-13T15:43:44.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.264+0000 7f2ea67fc640 1 -- 172.21.15.70:0/3907094926 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea000cb30 con 0x7f2eb012d330 2024-09-13T15:43:44.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.264+0000 7f2eb7ae9640 1 -- 172.21.15.70:0/3907094926 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb012ef90 con 0x7f2eb012d330 2024-09-13T15:43:44.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.264+0000 7f2ea67fc640 1 -- 172.21.15.70:0/3907094926 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea000cea0 con 0x7f2eb012d330 2024-09-13T15:43:44.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.265+0000 7f2ea67fc640 1 -- 172.21.15.70:0/3907094926 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 9) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f2ea0013470 con 0x7f2eb012d330 2024-09-13T15:43:44.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.266+0000 7f2eb7ae9640 1 -- 172.21.15.70:0/3907094926 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb012d840 con 0x7f2eb012d330 2024-09-13T15:43:44.273 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.266+0000 7f2ea67fc640 1 -- 172.21.15.70:0/3907094926 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f2ea005cd90 con 0x7f2eb012d330 2024-09-13T15:43:44.274 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.273+0000 7f2ea67fc640 1 -- 172.21.15.70:0/3907094926 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f2ea0060030 con 0x7f2eb012d330 2024-09-13T15:43:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.306+0000 7f14d53e9640 1 Processor -- start 2024-09-13T15:43:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.306+0000 7f14d53e9640 1 -- start start 2024-09-13T15:43:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.306+0000 7f14d53e9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d012d330 0x7f14d012d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.307+0000 7f14d53e9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f14d012dd80 0x7f14d0129e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.307+0000 7f14d53e9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d006d5d0 con 0x7f14d012a370 2024-09-13T15:43:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.307+0000 7f14d53e9640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00971b0 con 0x7f14d012d330 2024-09-13T15:43:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.307+0000 7f14d53e9640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00979f0 con 0x7f14d012dd80 2024-09-13T15:43:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.307+0000 7f14ce7fc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f14d012dd80 0x7f14d0129e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.307+0000 7f14ce7fc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f14d012dd80 0x7f14d0129e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:38238/0 (socket says 172.21.15.70:38238) 2024-09-13T15:43:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.307+0000 7f14ce7fc640 1 -- 172.21.15.70:0/2855118886 learned_addr learned my addr 172.21.15.70:0/2855118886 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.307+0000 7f14ceffd640 1 --2- 172.21.15.70:0/2855118886 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d012d330 0x7f14d012d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.307+0000 7f14ce7fc640 1 -- 172.21.15.70:0/2855118886 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d012d330 msgr2=0x7f14d012d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.307+0000 7f14ce7fc640 1 --2- 172.21.15.70:0/2855118886 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d012d330 0x7f14d012d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.307+0000 7f14ce7fc640 1 -- 172.21.15.70:0/2855118886 >> v1:172.21.15.70:6789/0 conn(0x7f14d012a370 legacy=0x7f14d0134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.307+0000 7f14ce7fc640 1 -- 172.21.15.70:0/2855118886 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d012e1a0 con 0x7f14d012dd80 2024-09-13T15:43:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.307+0000 7f14ceffd640 1 --2- 172.21.15.70:0/2855118886 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d012d330 0x7f14d012d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.308+0000 7f14ce7fc640 1 --2- 172.21.15.70:0/2855118886 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f14d012dd80 0x7f14d0129e30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f14c40098a0 tx=0x7f14c402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=249b526f101549ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.308+0000 7f14cdffb640 1 -- 172.21.15.70:0/2855118886 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c402ae60 con 0x7f14d012dd80 2024-09-13T15:43:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.308+0000 7f14cdffb640 1 -- 172.21.15.70:0/2855118886 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c4032440 con 0x7f14d012dd80 2024-09-13T15:43:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.308+0000 7f14cdffb640 1 -- 172.21.15.70:0/2855118886 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c40327d0 con 0x7f14d012dd80 2024-09-13T15:43:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.309+0000 7f14d53e9640 1 -- 172.21.15.70:0/2855118886 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f14d012dd80 msgr2=0x7f14d0129e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.309+0000 7f14d53e9640 1 --2- 172.21.15.70:0/2855118886 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f14d012dd80 0x7f14d0129e30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f14c40098a0 tx=0x7f14c402a110 comp rx=0 tx=0).stop 2024-09-13T15:43:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.309+0000 7f14d53e9640 1 -- 172.21.15.70:0/2855118886 shutdown_connections 2024-09-13T15:43:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.309+0000 7f14d53e9640 1 --2- 172.21.15.70:0/2855118886 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f14d012dd80 0x7f14d0129e30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.309+0000 7f14d53e9640 1 --2- 172.21.15.70:0/2855118886 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d012d330 0x7f14d012d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.309+0000 7f14d53e9640 1 -- 172.21.15.70:0/2855118886 >> 172.21.15.70:0/2855118886 conn(0x7f14d00a0060 msgr2=0x7f14d00a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:44.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.309+0000 7f14d53e9640 1 -- 172.21.15.70:0/2855118886 shutdown_connections 2024-09-13T15:43:44.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.309+0000 7f14d53e9640 1 -- 172.21.15.70:0/2855118886 wait complete. 2024-09-13T15:43:44.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.310+0000 7f14d53e9640 1 Processor -- start 2024-09-13T15:43:44.313 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.310+0000 7f14d53e9640 1 -- start start 2024-09-13T15:43:44.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.310+0000 7f14d53e9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d012a370 0x7f14d01be3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.310+0000 7f14d53e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14d012d330 0x7f14d01be900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.310+0000 7f14d53e9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f14d012dd80 0x7f14d0127060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.310+0000 7f14d53e9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d01292c0 con 0x7f14d012d330 2024-09-13T15:43:44.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.310+0000 7f14d53e9640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d008b480 con 0x7f14d012a370 2024-09-13T15:43:44.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.310+0000 7f14d53e9640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14d0128990 con 0x7f14d012dd80 2024-09-13T15:43:44.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.310+0000 7f14ceffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d012a370 0x7f14d01be3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.310+0000 7f14ceffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d012a370 0x7f14d01be3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:54728/0 (socket says 172.21.15.70:54728) 2024-09-13T15:43:44.315 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.310+0000 7f14ceffd640 1 -- 172.21.15.70:0/388537105 learned_addr learned my addr 172.21.15.70:0/388537105 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:44.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.310+0000 7f14cf7fe640 1 --2- 172.21.15.70:0/388537105 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f14d012dd80 0x7f14d0127060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.310+0000 7f14ce7fc640 1 --2- 172.21.15.70:0/388537105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14d012d330 0x7f14d01be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.311+0000 7f14ceffd640 1 -- 172.21.15.70:0/388537105 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f14d012dd80 msgr2=0x7f14d0127060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.311+0000 7f14ceffd640 1 --2- 172.21.15.70:0/388537105 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f14d012dd80 0x7f14d0127060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.311+0000 7f14ceffd640 1 -- 172.21.15.70:0/388537105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14d012d330 msgr2=0x7f14d01be900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.311+0000 7f14ceffd640 1 --2- 172.21.15.70:0/388537105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14d012d330 0x7f14d01be900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.311+0000 7f14ceffd640 1 -- 172.21.15.70:0/388537105 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c0011ca0 con 0x7f14d012a370 2024-09-13T15:43:44.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.311+0000 7f14ceffd640 1 --2- 172.21.15.70:0/388537105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d012a370 0x7f14d01be3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f14c00143b0 tx=0x7f14c0014870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:44.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.311+0000 7f14b77fe640 1 -- 172.21.15.70:0/388537105 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0016050 con 0x7f14d012a370 2024-09-13T15:43:44.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.311+0000 7f14b77fe640 1 -- 172.21.15.70:0/388537105 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c0017040 con 0x7f14d012a370 2024-09-13T15:43:44.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.311+0000 7f14d53e9640 1 -- 172.21.15.70:0/388537105 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c40094d0 con 0x7f14d012a370 2024-09-13T15:43:44.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.311+0000 7f14b77fe640 1 -- 172.21.15.70:0/388537105 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c00116e0 con 0x7f14d012a370 2024-09-13T15:43:44.316 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.311+0000 7f14d53e9640 1 -- 172.21.15.70:0/388537105 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d008b240 con 0x7f14d012a370 2024-09-13T15:43:44.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.312+0000 7f14b77fe640 1 -- 172.21.15.70:0/388537105 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 9) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f14c000f850 con 0x7f14d012a370 2024-09-13T15:43:44.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.312+0000 7f14d53e9640 1 -- 172.21.15.70:0/388537105 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d0098f30 con 0x7f14d012a370 2024-09-13T15:43:44.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.313+0000 7f14b77fe640 1 -- 172.21.15.70:0/388537105 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f14c0064870 con 0x7f14d012a370 2024-09-13T15:43:44.317 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.316+0000 7f14b77fe640 1 -- 172.21.15.70:0/388537105 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f14c0042560 con 0x7f14d012a370 2024-09-13T15:43:44.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.343+0000 7fe7e0a27640 1 Processor -- start 2024-09-13T15:43:44.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.343+0000 7fe7e0a27640 1 -- start start 2024-09-13T15:43:44.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.344+0000 7fe7e0a27640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7dc12d780 0x7fe7dc12db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.344+0000 7fe7e0a27640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe7dc06d3e0 0x7fe7dc135670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.344+0000 7fe7e0a27640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7dc12d330 con 0x7fe7dc06c8b0 2024-09-13T15:43:44.346 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.344+0000 7fe7e0a27640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7dc06dc70 con 0x7fe7dc06d3e0 2024-09-13T15:43:44.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.344+0000 7fe7e0a27640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc09d4c0 con 0x7fe7dc12d780 2024-09-13T15:43:44.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.344+0000 7fe7dad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe7dc06d3e0 0x7fe7dc135670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.344+0000 7fe7dad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe7dc06d3e0 0x7fe7dc135670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37470/0 (socket says 172.21.15.70:37470) 2024-09-13T15:43:44.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.344+0000 7fe7dad9d640 1 -- 172.21.15.70:0/2810903882 learned_addr learned my addr 172.21.15.70:0/2810903882 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:44.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.344+0000 7fe7da59c640 1 --2- 172.21.15.70:0/2810903882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7dc12d780 0x7fe7dc12db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.345+0000 7fe7dad9d640 1 -- 172.21.15.70:0/2810903882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7dc12d780 msgr2=0x7fe7dc12db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:44.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.345+0000 7fe7dad9d640 1 --2- 172.21.15.70:0/2810903882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7dc12d780 0x7fe7dc12db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.345+0000 7fe7dad9d640 1 -- 172.21.15.70:0/2810903882 >> v1:172.21.15.70:6789/0 conn(0x7fe7dc06c8b0 legacy=0x7fe7dc06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.345+0000 7fe7dad9d640 1 -- 172.21.15.70:0/2810903882 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7dc067710 con 0x7fe7dc06d3e0 2024-09-13T15:43:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.346+0000 7fe7dad9d640 1 --2- 172.21.15.70:0/2810903882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7dc06d3e0 0x7fe7dc135670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009e50 tx=0x7fe7d402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2984f5ea8a46c32c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.346+0000 7fe7d959a640 1 -- 172.21.15.70:0/2810903882 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4031600 con 0x7fe7dc06d3e0 2024-09-13T15:43:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.346+0000 7fe7d959a640 1 -- 172.21.15.70:0/2810903882 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d402d050 con 0x7fe7dc06d3e0 2024-09-13T15:43:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.346+0000 7fe7d959a640 1 -- 172.21.15.70:0/2810903882 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4031980 con 0x7fe7dc06d3e0 2024-09-13T15:43:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.347+0000 7fe7e0a27640 1 -- 172.21.15.70:0/2810903882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7dc06d3e0 msgr2=0x7fe7dc135670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.347+0000 7fe7e0a27640 1 --2- 172.21.15.70:0/2810903882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7dc06d3e0 0x7fe7dc135670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009e50 tx=0x7fe7d402a610 comp rx=0 tx=0).stop 2024-09-13T15:43:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.347+0000 7fe7e0a27640 1 -- 172.21.15.70:0/2810903882 shutdown_connections 2024-09-13T15:43:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.347+0000 7fe7e0a27640 1 --2- 172.21.15.70:0/2810903882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7dc06d3e0 0x7fe7dc135670 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.347+0000 7fe7e0a27640 1 --2- 172.21.15.70:0/2810903882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7dc12d780 0x7fe7dc12db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.347+0000 7fe7e0a27640 1 -- 172.21.15.70:0/2810903882 >> 172.21.15.70:0/2810903882 conn(0x7fe7dc0a02e0 msgr2=0x7fe7dc09c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:44.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.347+0000 7fe7e0a27640 1 -- 172.21.15.70:0/2810903882 shutdown_connections 2024-09-13T15:43:44.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.347+0000 7fe7e0a27640 1 -- 172.21.15.70:0/2810903882 wait complete. 2024-09-13T15:43:44.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.348+0000 7fe7e0a27640 1 Processor -- start 2024-09-13T15:43:44.350 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.348+0000 7fe7e0a27640 1 -- start start 2024-09-13T15:43:44.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.348+0000 7fe7e0a27640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7dc06c8b0 0x7fe7dc12ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.348+0000 7fe7e0a27640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe7dc06d3e0 0x7fe7dc12ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.348+0000 7fe7e0a27640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7dc12d780 0x7fe7dc12f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.352 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.348+0000 7fe7e0a27640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc130e40 con 0x7fe7dc06c8b0 2024-09-13T15:43:44.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.348+0000 7fe7e0a27640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc131040 con 0x7fe7dc12d780 2024-09-13T15:43:44.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.348+0000 7fe7e0a27640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc09a130 con 0x7fe7dc06d3e0 2024-09-13T15:43:44.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.348+0000 7fe7dad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7dc12d780 0x7fe7dc12f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.348+0000 7fe7dad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7dc12d780 0x7fe7dc12f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:54748/0 (socket says 172.21.15.70:54748) 2024-09-13T15:43:44.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.348+0000 7fe7da59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7dc06c8b0 0x7fe7dc12ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.348+0000 7fe7dad9d640 1 -- 172.21.15.70:0/2852275988 learned_addr learned my addr 172.21.15.70:0/2852275988 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:44.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.348+0000 7fe7d9d9b640 1 --2- 172.21.15.70:0/2852275988 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe7dc06d3e0 0x7fe7dc12ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.349+0000 7fe7dad9d640 1 -- 172.21.15.70:0/2852275988 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe7dc06d3e0 msgr2=0x7fe7dc12ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.349+0000 7fe7dad9d640 1 --2- 172.21.15.70:0/2852275988 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe7dc06d3e0 0x7fe7dc12ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.349+0000 7fe7dad9d640 1 -- 172.21.15.70:0/2852275988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7dc06c8b0 msgr2=0x7fe7dc12ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.349+0000 7fe7dad9d640 1 --2- 172.21.15.70:0/2852275988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7dc06c8b0 0x7fe7dc12ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.349+0000 7fe7dad9d640 1 -- 172.21.15.70:0/2852275988 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d4009b30 con 0x7fe7dc12d780 2024-09-13T15:43:44.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.349+0000 7fe7da59c640 1 --2- 172.21.15.70:0/2852275988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7dc06c8b0 0x7fe7dc12ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:44.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.349+0000 7fe7dad9d640 1 --2- 172.21.15.70:0/2852275988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7dc12d780 0x7fe7dc12f4b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe7d402ab30 tx=0x7fe7d4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:44.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.349+0000 7fe7caffd640 1 -- 172.21.15.70:0/2852275988 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d403d860 con 0x7fe7dc12d780 2024-09-13T15:43:44.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.349+0000 7fe7e0a27640 1 -- 172.21.15.70:0/2852275988 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7dc1322a0 con 0x7fe7dc12d780 2024-09-13T15:43:44.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.349+0000 7fe7caffd640 1 -- 172.21.15.70:0/2852275988 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d403d9c0 con 0x7fe7dc12d780 2024-09-13T15:43:44.354 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.349+0000 7fe7e0a27640 1 -- 172.21.15.70:0/2852275988 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7dc135020 con 0x7fe7dc12d780 2024-09-13T15:43:44.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.350+0000 7fe7caffd640 1 -- 172.21.15.70:0/2852275988 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d403dd70 con 0x7fe7dc12d780 2024-09-13T15:43:44.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.351+0000 7fe7caffd640 1 -- 172.21.15.70:0/2852275988 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 9) v1 ==== 55156+0+0 (secure 0 0 0) 0x7fe7d403b400 con 0x7fe7dc12d780 2024-09-13T15:43:44.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.351+0000 7fe7e0a27640 1 -- 172.21.15.70:0/2852275988 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c0005250 con 0x7fe7dc12d780 2024-09-13T15:43:44.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.351+0000 7fe7caffd640 1 -- 172.21.15.70:0/2852275988 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fe7d4082290 con 0x7fe7dc12d780 2024-09-13T15:43:44.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.355+0000 7fe7caffd640 1 -- 172.21.15.70:0/2852275988 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fe7d405e420 con 0x7fe7dc12d780 2024-09-13T15:43:44.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.383+0000 7f3195a41640 1 -- 172.21.15.70:0/2047086579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f319008d9f0 con 0x7f319006d6f0 2024-09-13T15:43:44.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.383+0000 7f3177fff640 1 -- 172.21.15.70:0/2047086579 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3184018030 con 0x7f319006d6f0 2024-09-13T15:43:44.385 INFO:teuthology.orchestra.run.smithi070.stdout:38654705683 2024-09-13T15:43:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.385+0000 7f3195a41640 1 -- 172.21.15.70:0/2047086579 >> v1:172.21.15.70:6816/1229510400 conn(0x7f317c0474d0 legacy=0x7f317c049990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.385+0000 7f3195a41640 1 -- 172.21.15.70:0/2047086579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319006d6f0 msgr2=0x7f319008b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.385+0000 7f3195a41640 1 --2- 172.21.15.70:0/2047086579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319006d6f0 0x7f319008b530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f318400c4d0 tx=0x7f318400c990 comp rx=0 tx=0).stop 2024-09-13T15:43:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.385+0000 7f3195a41640 1 -- 172.21.15.70:0/2047086579 shutdown_connections 2024-09-13T15:43:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.385+0000 7f3195a41640 1 --2- 172.21.15.70:0/2047086579 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f319012a810 0x7f319008ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.385+0000 7f3195a41640 1 --2- 172.21.15.70:0/2047086579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f319006d6f0 0x7f319008b530 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.385+0000 7f3195a41640 1 --2- 172.21.15.70:0/2047086579 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f319006cdd0 0x7f319008aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.385+0000 7f3195a41640 1 -- 172.21.15.70:0/2047086579 >> 172.21.15.70:0/2047086579 conn(0x7f31900973c0 msgr2=0x7f319009a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.385+0000 7f3195a41640 1 -- 172.21.15.70:0/2047086579 shutdown_connections 2024-09-13T15:43:44.387 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.385+0000 7f3195a41640 1 -- 172.21.15.70:0/2047086579 wait complete. 2024-09-13T15:43:44.397 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705683 for osd.2 2024-09-13T15:43:44.397 DEBUG:teuthology.parallel:result is None 2024-09-13T15:43:44.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.460+0000 7f14d53e9640 1 -- 172.21.15.70:0/388537105 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f14d008b240 con 0x7f14d012a370 2024-09-13T15:43:44.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.461+0000 7f14b77fe640 1 -- 172.21.15.70:0/388537105 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14d008b240 con 0x7f14d012a370 2024-09-13T15:43:44.462 INFO:teuthology.orchestra.run.smithi070.stdout:38654705683 2024-09-13T15:43:44.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.462+0000 7f14d53e9640 1 -- 172.21.15.70:0/388537105 >> v1:172.21.15.70:6816/1229510400 conn(0x7f14bc0474d0 legacy=0x7f14bc049990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.462+0000 7f14d53e9640 1 -- 172.21.15.70:0/388537105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d012a370 msgr2=0x7f14d01be3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.462+0000 7f14d53e9640 1 --2- 172.21.15.70:0/388537105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d012a370 0x7f14d01be3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f14c00143b0 tx=0x7f14c0014870 comp rx=0 tx=0).stop 2024-09-13T15:43:44.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.463+0000 7f14d53e9640 1 -- 172.21.15.70:0/388537105 shutdown_connections 2024-09-13T15:43:44.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.463+0000 7f14d53e9640 1 --2- 172.21.15.70:0/388537105 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f14d012dd80 0x7f14d0127060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.463+0000 7f14d53e9640 1 --2- 172.21.15.70:0/388537105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14d012d330 0x7f14d01be900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.465 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.463+0000 7f14d53e9640 1 --2- 172.21.15.70:0/388537105 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14d012a370 0x7f14d01be3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.465 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.463+0000 7f14d53e9640 1 -- 172.21.15.70:0/388537105 >> 172.21.15.70:0/388537105 conn(0x7f14d00a0060 msgr2=0x7f14d009b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:44.465 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.463+0000 7f14d53e9640 1 -- 172.21.15.70:0/388537105 shutdown_connections 2024-09-13T15:43:44.465 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.463+0000 7f14d53e9640 1 -- 172.21.15.70:0/388537105 wait complete. 2024-09-13T15:43:44.474 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705683 for osd.0 2024-09-13T15:43:44.474 DEBUG:teuthology.parallel:result is None 2024-09-13T15:43:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.520+0000 7f2eb7ae9640 1 -- 172.21.15.70:0/3907094926 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2eb01317d0 con 0x7f2eb012d330 2024-09-13T15:43:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.521+0000 7f2ea67fc640 1 -- 172.21.15.70:0/3907094926 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2ea0017020 con 0x7f2eb012d330 2024-09-13T15:43:44.522 INFO:teuthology.orchestra.run.smithi070.stdout:38654705683 2024-09-13T15:43:44.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.522+0000 7f2eb7ae9640 1 -- 172.21.15.70:0/3907094926 >> v1:172.21.15.70:6816/1229510400 conn(0x7f2e940474d0 legacy=0x7f2e94049990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.522+0000 7f2eb7ae9640 1 -- 172.21.15.70:0/3907094926 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f2eb012d330 msgr2=0x7f2eb0132e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.522+0000 7f2eb7ae9640 1 --2- 172.21.15.70:0/3907094926 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f2eb012d330 0x7f2eb0132e90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000e9e0 tx=0x7f2ea000eea0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.522+0000 7f2eb7ae9640 1 -- 172.21.15.70:0/3907094926 shutdown_connections 2024-09-13T15:43:44.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.522+0000 7f2eb7ae9640 1 --2- 172.21.15.70:0/3907094926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb012dd80 0x7f2eb01333d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.522+0000 7f2eb7ae9640 1 --2- 172.21.15.70:0/3907094926 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f2eb012d330 0x7f2eb0132e90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.522+0000 7f2eb7ae9640 1 --2- 172.21.15.70:0/3907094926 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2eb012a370 0x7f2eb0132950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.525 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.522+0000 7f2eb7ae9640 1 -- 172.21.15.70:0/3907094926 >> 172.21.15.70:0/3907094926 conn(0x7f2eb00a0060 msgr2=0x7f2eb0098640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:44.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.523+0000 7f2eb7ae9640 1 -- 172.21.15.70:0/3907094926 shutdown_connections 2024-09-13T15:43:44.526 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.523+0000 7f2eb7ae9640 1 -- 172.21.15.70:0/3907094926 wait complete. 2024-09-13T15:43:44.538 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705682 got 38654705683 for osd.3 2024-09-13T15:43:44.538 DEBUG:teuthology.parallel:result is None 2024-09-13T15:43:44.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.576+0000 7fe7e0a27640 1 -- 172.21.15.70:0/2852275988 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe7c00057d0 con 0x7fe7dc12d780 2024-09-13T15:43:44.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.576+0000 7fe7caffd640 1 -- 172.21.15.70:0/2852275988 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe7d4036070 con 0x7fe7dc12d780 2024-09-13T15:43:44.578 INFO:teuthology.orchestra.run.smithi070.stdout:38654705682 2024-09-13T15:43:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.578+0000 7fe7e0a27640 1 -- 172.21.15.70:0/2852275988 >> v1:172.21.15.70:6816/1229510400 conn(0x7fe7bc0474d0 legacy=0x7fe7bc049990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.578+0000 7fe7e0a27640 1 -- 172.21.15.70:0/2852275988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7dc12d780 msgr2=0x7fe7dc12f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.578+0000 7fe7e0a27640 1 --2- 172.21.15.70:0/2852275988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7dc12d780 0x7fe7dc12f4b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe7d402ab30 tx=0x7fe7d4004a40 comp rx=0 tx=0).stop 2024-09-13T15:43:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.578+0000 7fe7e0a27640 1 -- 172.21.15.70:0/2852275988 shutdown_connections 2024-09-13T15:43:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.578+0000 7fe7e0a27640 1 --2- 172.21.15.70:0/2852275988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe7dc12d780 0x7fe7dc12f4b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.578+0000 7fe7e0a27640 1 --2- 172.21.15.70:0/2852275988 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe7dc06d3e0 0x7fe7dc12ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.578+0000 7fe7e0a27640 1 --2- 172.21.15.70:0/2852275988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7dc06c8b0 0x7fe7dc12ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.578+0000 7fe7e0a27640 1 -- 172.21.15.70:0/2852275988 >> 172.21.15.70:0/2852275988 conn(0x7fe7dc0a02e0 msgr2=0x7fe7dc09b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.578+0000 7fe7e0a27640 1 -- 172.21.15.70:0/2852275988 shutdown_connections 2024-09-13T15:43:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.578+0000 7fe7e0a27640 1 -- 172.21.15.70:0/2852275988 wait complete. 2024-09-13T15:43:44.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705681 got 38654705682 for osd.1 2024-09-13T15:43:44.590 DEBUG:teuthology.parallel:result is None 2024-09-13T15:43:44.590 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T15:43:44.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T15:43:44.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.725+0000 7ff5ccd65640 1 Processor -- start 2024-09-13T15:43:44.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.725+0000 7ff5ccd65640 1 -- start start 2024-09-13T15:43:44.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.726+0000 7ff5ccd65640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5c806bb10 0x7ff5c806bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.726+0000 7ff5ccd65640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c806c430 0x7ff5c812d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.726+0000 7ff5ccd65640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c8097500 con 0x7ff5c812d930 2024-09-13T15:43:44.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.726+0000 7ff5ccd65640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c80979d0 con 0x7ff5c806bb10 2024-09-13T15:43:44.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.726+0000 7ff5ccd65640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c806c810 con 0x7ff5c806c430 2024-09-13T15:43:44.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.726+0000 7ff5c659c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5c806bb10 0x7ff5c806bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.726+0000 7ff5c6d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff5c812d930 0x7ff5c812ddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33634/0 (socket says 172.21.15.70:33634) 2024-09-13T15:43:44.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.726+0000 7ff5c5d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c806c430 0x7ff5c812d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.726+0000 7ff5c6d9d640 1 -- 172.21.15.70:0/588549363 learned_addr learned my addr 172.21.15.70:0/588549363 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:44.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.726+0000 7ff5c659c640 1 -- 172.21.15.70:0/588549363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c806c430 msgr2=0x7ff5c812d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:44.729 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.726+0000 7ff5c659c640 1 --2- 172.21.15.70:0/588549363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c806c430 0x7ff5c812d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.726+0000 7ff5c659c640 1 -- 172.21.15.70:0/588549363 >> v1:172.21.15.70:6789/0 conn(0x7ff5c812d930 legacy=0x7ff5c812ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.726+0000 7ff5c659c640 1 -- 172.21.15.70:0/588549363 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c806d100 con 0x7ff5c806bb10 2024-09-13T15:43:44.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.726+0000 7ff5c559a640 1 -- 172.21.15.70:0/588549363 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 990513739 0 0) 0x7ff5c8097500 con 0x7ff5c812d930 2024-09-13T15:43:44.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.726+0000 7ff5c5d9b640 1 --2- 172.21.15.70:0/588549363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c806c430 0x7ff5c812d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:44.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.727+0000 7ff5c659c640 1 --2- 172.21.15.70:0/588549363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c806bb10 0x7ff5c806bef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009d90 tx=0x7ff5b802a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d35e36fa52b93b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:44.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.727+0000 7ff5c559a640 1 -- 172.21.15.70:0/588549363 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8007740 con 0x7ff5c806bb10 2024-09-13T15:43:44.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.727+0000 7ff5c559a640 1 -- 172.21.15.70:0/588549363 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b802d050 con 0x7ff5c806bb10 2024-09-13T15:43:44.730 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.727+0000 7ff5c559a640 1 -- 172.21.15.70:0/588549363 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8007ac0 con 0x7ff5c806bb10 2024-09-13T15:43:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.728+0000 7ff5ccd65640 1 -- 172.21.15.70:0/588549363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c806bb10 msgr2=0x7ff5c806bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.728+0000 7ff5ccd65640 1 --2- 172.21.15.70:0/588549363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c806bb10 0x7ff5c806bef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009d90 tx=0x7ff5b802a540 comp rx=0 tx=0).stop 2024-09-13T15:43:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.728+0000 7ff5ccd65640 1 -- 172.21.15.70:0/588549363 shutdown_connections 2024-09-13T15:43:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.728+0000 7ff5ccd65640 1 --2- 172.21.15.70:0/588549363 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c806c430 0x7ff5c812d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.728+0000 7ff5ccd65640 1 --2- 172.21.15.70:0/588549363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c806bb10 0x7ff5c806bef0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.728+0000 7ff5ccd65640 1 -- 172.21.15.70:0/588549363 >> 172.21.15.70:0/588549363 conn(0x7ff5c809fe20 msgr2=0x7ff5c80a0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.728+0000 7ff5ccd65640 1 -- 172.21.15.70:0/588549363 shutdown_connections 2024-09-13T15:43:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.728+0000 7ff5ccd65640 1 -- 172.21.15.70:0/588549363 wait complete. 2024-09-13T15:43:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5ccd65640 1 Processor -- start 2024-09-13T15:43:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5ccd65640 1 -- start start 2024-09-13T15:43:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5ccd65640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5c806bb10 0x7ff5c81bea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5ccd65640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c806c430 0x7ff5c81bef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5ccd65640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c812d930 0x7ff5c81bf480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5ccd65640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c8131f90 con 0x7ff5c806c430 2024-09-13T15:43:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5ccd65640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c8132250 con 0x7ff5c812d930 2024-09-13T15:43:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5ccd65640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c8130e10 con 0x7ff5c806bb10 2024-09-13T15:43:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5c659c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5c806bb10 0x7ff5c81bea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5c5d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c806c430 0x7ff5c81bef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5c6d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c812d930 0x7ff5c81bf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5c5d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c806c430 0x7ff5c81bef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57884/0 (socket says 172.21.15.70:57884) 2024-09-13T15:43:44.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5c5d9b640 1 -- 172.21.15.70:0/3748729124 learned_addr learned my addr 172.21.15.70:0/3748729124 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:44.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5c5d9b640 1 -- 172.21.15.70:0/3748729124 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5c806bb10 msgr2=0x7ff5c81bea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5c5d9b640 1 --2- 172.21.15.70:0/3748729124 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5c806bb10 0x7ff5c81bea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5c5d9b640 1 -- 172.21.15.70:0/3748729124 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c812d930 msgr2=0x7ff5c81bf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.729+0000 7ff5c5d9b640 1 --2- 172.21.15.70:0/3748729124 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c812d930 0x7ff5c81bf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.730+0000 7ff5c5d9b640 1 -- 172.21.15.70:0/3748729124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc009400 con 0x7ff5c806c430 2024-09-13T15:43:44.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.730+0000 7ff5c6d9d640 1 --2- 172.21.15.70:0/3748729124 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c812d930 0x7ff5c81bf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:44.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.730+0000 7ff5c5d9b640 1 --2- 172.21.15.70:0/3748729124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c806c430 0x7ff5c81bef40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00ebe0 tx=0x7ff5bc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:44.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.730+0000 7ff5b6ffd640 1 -- 172.21.15.70:0/3748729124 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc009760 con 0x7ff5c806c430 2024-09-13T15:43:44.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.730+0000 7ff5b6ffd640 1 -- 172.21.15.70:0/3748729124 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc00cc30 con 0x7ff5c806c430 2024-09-13T15:43:44.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.730+0000 7ff5ccd65640 1 -- 172.21.15.70:0/3748729124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b8009a70 con 0x7ff5c806c430 2024-09-13T15:43:44.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.730+0000 7ff5b6ffd640 1 -- 172.21.15.70:0/3748729124 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc00eee0 con 0x7ff5c806c430 2024-09-13T15:43:44.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.730+0000 7ff5ccd65640 1 -- 172.21.15.70:0/3748729124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c81c1f00 con 0x7ff5c806c430 2024-09-13T15:43:44.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.731+0000 7ff5b6ffd640 1 -- 172.21.15.70:0/3748729124 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 9) v1 ==== 55156+0+0 (secure 0 0 0) 0x7ff5bc013470 con 0x7ff5c806c430 2024-09-13T15:43:44.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.731+0000 7ff5ccd65640 1 -- 172.21.15.70:0/3748729124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c806bef0 con 0x7ff5c806c430 2024-09-13T15:43:44.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.731+0000 7ff5b6ffd640 1 -- 172.21.15.70:0/3748729124 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3023+0+0 (secure 0 0 0) 0x7ff5bc05cb30 con 0x7ff5c806c430 2024-09-13T15:43:44.738 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.735+0000 7ff5b6ffd640 1 -- 172.21.15.70:0/3748729124 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7ff5bc03a7c0 con 0x7ff5c806c430 2024-09-13T15:43:44.867 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.866+0000 7ff5ccd65640 1 -- 172.21.15.70:0/3748729124 --> v1:172.21.15.70:6816/1229510400 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5c8131570 con 0x7ff5a80474d0 2024-09-13T15:43:44.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.867+0000 7ff5b6ffd640 1 -- 172.21.15.70:0/3748729124 <== mgr.24114 v1:172.21.15.70:6816/1229510400 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40155 (unknown 2965378022 0 1071254639) 0x7ff5c8131570 con 0x7ff5a80474d0 2024-09-13T15:43:44.869 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:43:44.869 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-13T15:43:44.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.869+0000 7ff5ccd65640 1 -- 172.21.15.70:0/3748729124 >> v1:172.21.15.70:6816/1229510400 conn(0x7ff5a80474d0 legacy=0x7ff5a8049990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.869+0000 7ff5ccd65640 1 -- 172.21.15.70:0/3748729124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c806c430 msgr2=0x7ff5c81bef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.869+0000 7ff5ccd65640 1 --2- 172.21.15.70:0/3748729124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c806c430 0x7ff5c81bef40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00ebe0 tx=0x7ff5bc00c520 comp rx=0 tx=0).stop 2024-09-13T15:43:44.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.869+0000 7ff5ccd65640 1 -- 172.21.15.70:0/3748729124 shutdown_connections 2024-09-13T15:43:44.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.869+0000 7ff5ccd65640 1 --2- 172.21.15.70:0/3748729124 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5c812d930 0x7ff5c81bf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.869+0000 7ff5ccd65640 1 --2- 172.21.15.70:0/3748729124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c806c430 0x7ff5c81bef40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.869+0000 7ff5ccd65640 1 --2- 172.21.15.70:0/3748729124 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5c806bb10 0x7ff5c81bea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.869+0000 7ff5ccd65640 1 -- 172.21.15.70:0/3748729124 >> 172.21.15.70:0/3748729124 conn(0x7ff5c809fe20 msgr2=0x7ff5c8127f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:44.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.869+0000 7ff5ccd65640 1 -- 172.21.15.70:0/3748729124 shutdown_connections 2024-09-13T15:43:44.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.869+0000 7ff5ccd65640 1 -- 172.21.15.70:0/3748729124 wait complete. 2024-09-13T15:43:44.881 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-13T15:43:43.537063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1207872,"kb_used_data":1808,"kb_used_omap":6,"kb_used_meta":1206009,"kb_avail":376279488,"statfs":{"total":386547056640,"available":385310195712,"internally_reserved":0,"allocated":1851392,"data_stored":1373910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1234953568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.999406"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.658282+0000","last_change":"2024-09-13T15:42:29.720527+0000","last_active":"2024-09-13T15:42:34.658282+0000","last_peered":"2024-09-13T15:42:34.658282+0000","last_clean":"2024-09-13T15:42:34.658282+0000","last_became_active":"2024-09-13T15:42:27.716998+0000","last_became_peered":"2024-09-13T15:42:27.716998+0000","last_unstale":"2024-09-13T15:42:34.658282+0000","last_undegraded":"2024-09-13T15:42:34.658282+0000","last_fullsized":"2024-09-13T15:42:34.658282+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:22.475936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035985100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.766593+0000","last_change":"2024-09-13T15:42:29.717707+0000","last_active":"2024-09-13T15:42:34.766593+0000","last_peered":"2024-09-13T15:42:34.766593+0000","last_clean":"2024-09-13T15:42:34.766593+0000","last_became_active":"2024-09-13T15:42:27.718860+0000","last_became_peered":"2024-09-13T15:42:27.718860+0000","last_unstale":"2024-09-13T15:42:34.766593+0000","last_undegraded":"2024-09-13T15:42:34.766593+0000","last_fullsized":"2024-09-13T15:42:34.766593+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:19.555928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054553999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.659109+0000","last_change":"2024-09-13T15:42:29.721358+0000","last_active":"2024-09-13T15:42:34.659109+0000","last_peered":"2024-09-13T15:42:34.659109+0000","last_clean":"2024-09-13T15:42:34.659109+0000","last_became_active":"2024-09-13T15:42:27.717168+0000","last_became_peered":"2024-09-13T15:42:27.717168+0000","last_unstale":"2024-09-13T15:42:34.659109+0000","last_undegraded":"2024-09-13T15:42:34.659109+0000","last_fullsized":"2024-09-13T15:42:34.659109+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:15.334763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011766770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.766593+0000","last_change":"2024-09-13T15:42:29.717708+0000","last_active":"2024-09-13T15:42:34.766593+0000","last_peered":"2024-09-13T15:42:34.766593+0000","last_clean":"2024-09-13T15:42:34.766593+0000","last_became_active":"2024-09-13T15:42:27.719566+0000","last_became_peered":"2024-09-13T15:42:27.719566+0000","last_unstale":"2024-09-13T15:42:34.766593+0000","last_undegraded":"2024-09-13T15:42:34.766593+0000","last_fullsized":"2024-09-13T15:42:34.766593+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:59.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024452499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.659282+0000","last_change":"2024-09-13T15:42:29.721120+0000","last_active":"2024-09-13T15:42:34.659282+0000","last_peered":"2024-09-13T15:42:34.659282+0000","last_clean":"2024-09-13T15:42:34.659282+0000","last_became_active":"2024-09-13T15:42:27.718029+0000","last_became_peered":"2024-09-13T15:42:27.718029+0000","last_unstale":"2024-09-13T15:42:34.659282+0000","last_undegraded":"2024-09-13T15:42:34.659282+0000","last_fullsized":"2024-09-13T15:42:34.659282+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:44.778809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094542300000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.658156+0000","last_change":"2024-09-13T15:42:29.717606+0000","last_active":"2024-09-13T15:42:34.658156+0000","last_peered":"2024-09-13T15:42:34.658156+0000","last_clean":"2024-09-13T15:42:34.658156+0000","last_became_active":"2024-09-13T15:42:27.715108+0000","last_became_peered":"2024-09-13T15:42:27.715108+0000","last_unstale":"2024-09-13T15:42:34.658156+0000","last_undegraded":"2024-09-13T15:42:34.658156+0000","last_fullsized":"2024-09-13T15:42:34.658156+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:34.509753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089033199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.658535+0000","last_change":"2024-09-13T15:42:29.720489+0000","last_active":"2024-09-13T15:42:34.658535+0000","last_peered":"2024-09-13T15:42:34.658535+0000","last_clean":"2024-09-13T15:42:34.658535+0000","last_became_active":"2024-09-13T15:42:27.718347+0000","last_became_peered":"2024-09-13T15:42:27.718347+0000","last_unstale":"2024-09-13T15:42:34.658535+0000","last_undegraded":"2024-09-13T15:42:34.658535+0000","last_fullsized":"2024-09-13T15:42:34.658535+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:16.064731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034895899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.658675+0000","last_change":"2024-09-13T15:42:29.721205+0000","last_active":"2024-09-13T15:42:34.658675+0000","last_peered":"2024-09-13T15:42:34.658675+0000","last_clean":"2024-09-13T15:42:34.658675+0000","last_became_active":"2024-09-13T15:42:27.720033+0000","last_became_peered":"2024-09-13T15:42:27.720033+0000","last_unstale":"2024-09-13T15:42:34.658675+0000","last_undegraded":"2024-09-13T15:42:34.658675+0000","last_fullsized":"2024-09-13T15:42:34.658675+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:36.892284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276560000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":83,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.766676+0000","last_change":"2024-09-13T15:42:22.700543+0000","last_active":"2024-09-13T15:42:34.766676+0000","last_peered":"2024-09-13T15:42:34.766676+0000","last_clean":"2024-09-13T15:42:34.766676+0000","last_became_active":"2024-09-13T15:42:22.700226+0000","last_became_peered":"2024-09-13T15:42:22.700226+0000","last_unstale":"2024-09-13T15:42:34.766676+0000","last_undegraded":"2024-09-13T15:42:34.766676+0000","last_fullsized":"2024-09-13T15:42:34.766676+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_clean_scrub_stamp":"2024-09-13T15:42:21.689345+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:02.263603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705683,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301680,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070160,"statfs":{"total":96636764160,"available":96327843840,"internally_reserved":0,"allocated":167936,"data_stored":48303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 15:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.311,"15min":0.311},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.45700000000000002}]},{"osd":1,"last update":"Fri Sep 13 15:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.314,"15min":0.314},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Sep 13 15:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.442}]}]},{"osd":2,"up_from":9,"seq":38654705683,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301676,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070164,"statfs":{"total":96636764160,"available":96327847936,"internally_reserved":0,"allocated":163840,"data_stored":48284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 15:43:22 2024","interfaces":[{"interface":"back","average":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"min":{"1min":0.154,"5min":0.154,"15min":0.154},"max":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"last":0.65700000000000003}]},{"osd":1,"last update":"Fri Sep 13 15:43:22 2024","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.42899999999999999}]},{"osd":3,"last update":"Fri Sep 13 15:43:22 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.378}]}]},{"osd":1,"up_from":9,"seq":38654705682,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302260,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069580,"statfs":{"total":96636764160,"available":96327249920,"internally_reserved":0,"allocated":761856,"data_stored":638671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 15:43:22 2024","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Sep 13 15:43:22 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.36599999999999999}]},{"osd":3,"last update":"Fri Sep 13 15:43:22 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.35499999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705683,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302256,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069584,"statfs":{"total":96636764160,"available":96327254016,"internally_reserved":0,"allocated":757760,"data_stored":638652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 15:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.307},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.32100000000000001}]},{"osd":2,"last update":"Fri Sep 13 15:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.35899999999999999}]},{"osd":3,"last update":"Fri Sep 13 15:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.34100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T15:43:44.882 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T15:43:44.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.978+0000 7f0d3c1ed640 1 Processor -- start 2024-09-13T15:43:44.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.978+0000 7f0d3c1ed640 1 -- start start 2024-09-13T15:43:44.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.978+0000 7f0d3c1ed640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d3406bb10 0x7f0d3406bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.978+0000 7f0d3c1ed640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0d3406c430 0x7f0d3412d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.978+0000 7f0d3c1ed640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d34097940 con 0x7f0d3412d930 2024-09-13T15:43:44.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.978+0000 7f0d3c1ed640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d34097500 con 0x7f0d3406c430 2024-09-13T15:43:44.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.978+0000 7f0d3c1ed640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3406c810 con 0x7f0d3406bb10 2024-09-13T15:43:44.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.978+0000 7f0d39f89640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d3406bb10 0x7f0d3406bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.980 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.978+0000 7f0d39788640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0d3406c430 0x7f0d3412d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.978+0000 7f0d3a78a640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0d3412d930 0x7f0d3412dda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33646/0 (socket says 172.21.15.70:33646) 2024-09-13T15:43:44.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.978+0000 7f0d39f89640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d3406bb10 0x7f0d3406bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:40586/0 (socket says 172.21.15.70:40586) 2024-09-13T15:43:44.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.978+0000 7f0d3a78a640 1 -- 172.21.15.70:0/526952661 learned_addr learned my addr 172.21.15.70:0/526952661 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:44.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.979+0000 7f0d39788640 1 -- 172.21.15.70:0/526952661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d3406bb10 msgr2=0x7f0d3406bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:44.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.979+0000 7f0d39788640 1 --2- 172.21.15.70:0/526952661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d3406bb10 0x7f0d3406bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.979+0000 7f0d39788640 1 -- 172.21.15.70:0/526952661 >> v1:172.21.15.70:6789/0 conn(0x7f0d3412d930 legacy=0x7f0d3412dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.979+0000 7f0d39788640 1 -- 172.21.15.70:0/526952661 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3406d100 con 0x7f0d3406c430 2024-09-13T15:43:44.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.979+0000 7f0d38f87640 1 -- 172.21.15.70:0/526952661 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970540003 0 0) 0x7f0d34097940 con 0x7f0d3412d930 2024-09-13T15:43:44.981 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.979+0000 7f0d39f89640 1 --2- 172.21.15.70:0/526952661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d3406bb10 0x7f0d3406bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:44.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.979+0000 7f0d39788640 1 --2- 172.21.15.70:0/526952661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d3406c430 0x7f0d3412d3f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d280097a0 tx=0x7f0d2802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a5c45733644533 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:44.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.979+0000 7f0d38f87640 1 -- 172.21.15.70:0/526952661 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2802ae50 con 0x7f0d3406c430 2024-09-13T15:43:44.984 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.980+0000 7f0d38f87640 1 -- 172.21.15.70:0/526952661 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d28032440 con 0x7f0d3406c430 2024-09-13T15:43:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.980+0000 7f0d38f87640 1 -- 172.21.15.70:0/526952661 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d280327d0 con 0x7f0d3406c430 2024-09-13T15:43:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.980+0000 7f0d3c1ed640 1 -- 172.21.15.70:0/526952661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d3406c430 msgr2=0x7f0d3412d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.980+0000 7f0d3c1ed640 1 --2- 172.21.15.70:0/526952661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d3406c430 0x7f0d3412d3f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d280097a0 tx=0x7f0d2802a100 comp rx=0 tx=0).stop 2024-09-13T15:43:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.980+0000 7f0d3c1ed640 1 -- 172.21.15.70:0/526952661 shutdown_connections 2024-09-13T15:43:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.980+0000 7f0d3c1ed640 1 --2- 172.21.15.70:0/526952661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d3406c430 0x7f0d3412d3f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.980+0000 7f0d3c1ed640 1 --2- 172.21.15.70:0/526952661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d3406bb10 0x7f0d3406bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.980+0000 7f0d3c1ed640 1 -- 172.21.15.70:0/526952661 >> 172.21.15.70:0/526952661 conn(0x7f0d3409fe20 msgr2=0x7f0d340a0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.980+0000 7f0d3c1ed640 1 -- 172.21.15.70:0/526952661 shutdown_connections 2024-09-13T15:43:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.980+0000 7f0d3c1ed640 1 -- 172.21.15.70:0/526952661 wait complete. 2024-09-13T15:43:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.981+0000 7f0d3c1ed640 1 Processor -- start 2024-09-13T15:43:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.981+0000 7f0d3c1ed640 1 -- start start 2024-09-13T15:43:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.981+0000 7f0d3c1ed640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d3406bb10 0x7f0d34093730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.981+0000 7f0d3c1ed640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0d3406c430 0x7f0d34093c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.981+0000 7f0d3c1ed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d3412d930 0x7f0d340941b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.981+0000 7f0d3c1ed640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34095960 con 0x7f0d3412d930 2024-09-13T15:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.981+0000 7f0d3c1ed640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340961b0 con 0x7f0d3406bb10 2024-09-13T15:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.981+0000 7f0d3c1ed640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34096d60 con 0x7f0d3406c430 2024-09-13T15:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.981+0000 7f0d3a78a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d3412d930 0x7f0d340941b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.981+0000 7f0d39f89640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d3406bb10 0x7f0d34093730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.981+0000 7f0d39788640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0d3406c430 0x7f0d34093c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.981+0000 7f0d3a78a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d3412d930 0x7f0d340941b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57906/0 (socket says 172.21.15.70:57906) 2024-09-13T15:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.981+0000 7f0d39f89640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d3406bb10 0x7f0d34093730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:40598/0 (socket says 172.21.15.70:40598) 2024-09-13T15:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.981+0000 7f0d3a78a640 1 -- 172.21.15.70:0/1965364337 learned_addr learned my addr 172.21.15.70:0/1965364337 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.982+0000 7f0d39f89640 1 -- 172.21.15.70:0/1965364337 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0d3406c430 msgr2=0x7f0d34093c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.982+0000 7f0d39f89640 1 --2- 172.21.15.70:0/1965364337 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0d3406c430 0x7f0d34093c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.982+0000 7f0d39f89640 1 -- 172.21.15.70:0/1965364337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d3412d930 msgr2=0x7f0d340941b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.982+0000 7f0d39f89640 1 --2- 172.21.15.70:0/1965364337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d3412d930 0x7f0d340941b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.982+0000 7f0d39f89640 1 -- 172.21.15.70:0/1965364337 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d24009a70 con 0x7f0d3406bb10 2024-09-13T15:43:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.982+0000 7f0d3a78a640 1 --2- 172.21.15.70:0/1965364337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d3412d930 0x7f0d340941b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.982+0000 7f0d39788640 1 --2- 172.21.15.70:0/1965364337 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0d3406c430 0x7f0d34093c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.982+0000 7f0d39f89640 1 --2- 172.21.15.70:0/1965364337 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d3406bb10 0x7f0d34093730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400efc0 tx=0x7f0d2400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.982+0000 7f0d327fc640 1 -- 172.21.15.70:0/1965364337 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24014680 con 0x7f0d3406bb10 2024-09-13T15:43:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.982+0000 7f0d327fc640 1 -- 172.21.15.70:0/1965364337 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2400f040 con 0x7f0d3406bb10 2024-09-13T15:43:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.982+0000 7f0d3c1ed640 1 -- 172.21.15.70:0/1965364337 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d28009400 con 0x7f0d3406bb10 2024-09-13T15:43:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.982+0000 7f0d327fc640 1 -- 172.21.15.70:0/1965364337 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24014a20 con 0x7f0d3406bb10 2024-09-13T15:43:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.982+0000 7f0d3c1ed640 1 -- 172.21.15.70:0/1965364337 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d34135130 con 0x7f0d3406bb10 2024-09-13T15:43:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.983+0000 7f0d3c1ed640 1 -- 172.21.15.70:0/1965364337 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3406bef0 con 0x7f0d3406bb10 2024-09-13T15:43:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.983+0000 7f0d327fc640 1 -- 172.21.15.70:0/1965364337 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 9) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f0d24014b80 con 0x7f0d3406bb10 2024-09-13T15:43:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.983+0000 7f0d327fc640 1 -- 172.21.15.70:0/1965364337 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f0d2405d750 con 0x7f0d3406bb10 2024-09-13T15:43:44.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:44.987+0000 7f0d327fc640 1 -- 172.21.15.70:0/1965364337 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f0d2403b440 con 0x7f0d3406bb10 2024-09-13T15:43:45.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.119+0000 7f0d3c1ed640 1 -- 172.21.15.70:0/1965364337 --> v1:172.21.15.70:6816/1229510400 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d3406a6b0 con 0x7f0d200474d0 2024-09-13T15:43:45.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.120+0000 7f0d327fc640 1 -- 172.21.15.70:0/1965364337 <== mgr.24114 v1:172.21.15.70:6816/1229510400 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+40155 (unknown 2965378022 0 1071254639) 0x7f0d3406a6b0 con 0x7f0d200474d0 2024-09-13T15:43:45.122 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:43:45.122 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-13T15:43:45.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.122+0000 7f0d3c1ed640 1 -- 172.21.15.70:0/1965364337 >> v1:172.21.15.70:6816/1229510400 conn(0x7f0d200474d0 legacy=0x7f0d20049990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.122+0000 7f0d3c1ed640 1 -- 172.21.15.70:0/1965364337 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d3406bb10 msgr2=0x7f0d34093730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.122+0000 7f0d3c1ed640 1 --2- 172.21.15.70:0/1965364337 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d3406bb10 0x7f0d34093730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0d2400efc0 tx=0x7f0d2400c490 comp rx=0 tx=0).stop 2024-09-13T15:43:45.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.122+0000 7f0d3c1ed640 1 -- 172.21.15.70:0/1965364337 shutdown_connections 2024-09-13T15:43:45.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.122+0000 7f0d3c1ed640 1 --2- 172.21.15.70:0/1965364337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d3412d930 0x7f0d340941b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.122+0000 7f0d3c1ed640 1 --2- 172.21.15.70:0/1965364337 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0d3406c430 0x7f0d34093c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.122+0000 7f0d3c1ed640 1 --2- 172.21.15.70:0/1965364337 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d3406bb10 0x7f0d34093730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.122+0000 7f0d3c1ed640 1 -- 172.21.15.70:0/1965364337 >> 172.21.15.70:0/1965364337 conn(0x7f0d3409fe20 msgr2=0x7f0d3412b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:45.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.122+0000 7f0d3c1ed640 1 -- 172.21.15.70:0/1965364337 shutdown_connections 2024-09-13T15:43:45.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.122+0000 7f0d3c1ed640 1 -- 172.21.15.70:0/1965364337 wait complete. 2024-09-13T15:43:45.134 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-13T15:43:43.537063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1207872,"kb_used_data":1808,"kb_used_omap":6,"kb_used_meta":1206009,"kb_avail":376279488,"statfs":{"total":386547056640,"available":385310195712,"internally_reserved":0,"allocated":1851392,"data_stored":1373910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6816,"internal_metadata":1234953568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.999406"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.658282+0000","last_change":"2024-09-13T15:42:29.720527+0000","last_active":"2024-09-13T15:42:34.658282+0000","last_peered":"2024-09-13T15:42:34.658282+0000","last_clean":"2024-09-13T15:42:34.658282+0000","last_became_active":"2024-09-13T15:42:27.716998+0000","last_became_peered":"2024-09-13T15:42:27.716998+0000","last_unstale":"2024-09-13T15:42:34.658282+0000","last_undegraded":"2024-09-13T15:42:34.658282+0000","last_fullsized":"2024-09-13T15:42:34.658282+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:37:22.475936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035985100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.766593+0000","last_change":"2024-09-13T15:42:29.717707+0000","last_active":"2024-09-13T15:42:34.766593+0000","last_peered":"2024-09-13T15:42:34.766593+0000","last_clean":"2024-09-13T15:42:34.766593+0000","last_became_active":"2024-09-13T15:42:27.718860+0000","last_became_peered":"2024-09-13T15:42:27.718860+0000","last_unstale":"2024-09-13T15:42:34.766593+0000","last_undegraded":"2024-09-13T15:42:34.766593+0000","last_fullsized":"2024-09-13T15:42:34.766593+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:19.555928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054553999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.659109+0000","last_change":"2024-09-13T15:42:29.721358+0000","last_active":"2024-09-13T15:42:34.659109+0000","last_peered":"2024-09-13T15:42:34.659109+0000","last_clean":"2024-09-13T15:42:34.659109+0000","last_became_active":"2024-09-13T15:42:27.717168+0000","last_became_peered":"2024-09-13T15:42:27.717168+0000","last_unstale":"2024-09-13T15:42:34.659109+0000","last_undegraded":"2024-09-13T15:42:34.659109+0000","last_fullsized":"2024-09-13T15:42:34.659109+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:15.334763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011766770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.766593+0000","last_change":"2024-09-13T15:42:29.717708+0000","last_active":"2024-09-13T15:42:34.766593+0000","last_peered":"2024-09-13T15:42:34.766593+0000","last_clean":"2024-09-13T15:42:34.766593+0000","last_became_active":"2024-09-13T15:42:27.719566+0000","last_became_peered":"2024-09-13T15:42:27.719566+0000","last_unstale":"2024-09-13T15:42:34.766593+0000","last_undegraded":"2024-09-13T15:42:34.766593+0000","last_fullsized":"2024-09-13T15:42:34.766593+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:59.493169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024452499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":20,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.659282+0000","last_change":"2024-09-13T15:42:29.721120+0000","last_active":"2024-09-13T15:42:34.659282+0000","last_peered":"2024-09-13T15:42:34.659282+0000","last_clean":"2024-09-13T15:42:34.659282+0000","last_became_active":"2024-09-13T15:42:27.718029+0000","last_became_peered":"2024-09-13T15:42:27.718029+0000","last_unstale":"2024-09-13T15:42:34.659282+0000","last_undegraded":"2024-09-13T15:42:34.659282+0000","last_fullsized":"2024-09-13T15:42:34.659282+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:44.778809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094542300000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.658156+0000","last_change":"2024-09-13T15:42:29.717606+0000","last_active":"2024-09-13T15:42:34.658156+0000","last_peered":"2024-09-13T15:42:34.658156+0000","last_clean":"2024-09-13T15:42:34.658156+0000","last_became_active":"2024-09-13T15:42:27.715108+0000","last_became_peered":"2024-09-13T15:42:27.715108+0000","last_unstale":"2024-09-13T15:42:34.658156+0000","last_undegraded":"2024-09-13T15:42:34.658156+0000","last_fullsized":"2024-09-13T15:42:34.658156+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:34.509753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089033199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.658535+0000","last_change":"2024-09-13T15:42:29.720489+0000","last_active":"2024-09-13T15:42:34.658535+0000","last_peered":"2024-09-13T15:42:34.658535+0000","last_clean":"2024-09-13T15:42:34.658535+0000","last_became_active":"2024-09-13T15:42:27.718347+0000","last_became_peered":"2024-09-13T15:42:27.718347+0000","last_unstale":"2024-09-13T15:42:34.658535+0000","last_undegraded":"2024-09-13T15:42:34.658535+0000","last_fullsized":"2024-09-13T15:42:34.658535+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:16.064731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034895899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.658675+0000","last_change":"2024-09-13T15:42:29.721205+0000","last_active":"2024-09-13T15:42:34.658675+0000","last_peered":"2024-09-13T15:42:34.658675+0000","last_clean":"2024-09-13T15:42:34.658675+0000","last_became_active":"2024-09-13T15:42:27.720033+0000","last_became_peered":"2024-09-13T15:42:27.720033+0000","last_unstale":"2024-09-13T15:42:34.658675+0000","last_undegraded":"2024-09-13T15:42:34.658675+0000","last_fullsized":"2024-09-13T15:42:34.658675+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:36.892284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276560000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":83,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T15:42:34.766676+0000","last_change":"2024-09-13T15:42:22.700543+0000","last_active":"2024-09-13T15:42:34.766676+0000","last_peered":"2024-09-13T15:42:34.766676+0000","last_clean":"2024-09-13T15:42:34.766676+0000","last_became_active":"2024-09-13T15:42:22.700226+0000","last_became_peered":"2024-09-13T15:42:22.700226+0000","last_unstale":"2024-09-13T15:42:34.766676+0000","last_undegraded":"2024-09-13T15:42:34.766676+0000","last_fullsized":"2024-09-13T15:42:34.766676+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_clean_scrub_stamp":"2024-09-13T15:42:21.689345+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:02.263603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705683,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301680,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070160,"statfs":{"total":96636764160,"available":96327843840,"internally_reserved":0,"allocated":167936,"data_stored":48303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 15:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.311,"15min":0.311},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.45700000000000002}]},{"osd":1,"last update":"Fri Sep 13 15:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.314,"15min":0.314},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Sep 13 15:43:21 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.442}]}]},{"osd":2,"up_from":9,"seq":38654705683,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301676,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070164,"statfs":{"total":96636764160,"available":96327847936,"internally_reserved":0,"allocated":163840,"data_stored":48284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 15:43:22 2024","interfaces":[{"interface":"back","average":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"min":{"1min":0.154,"5min":0.154,"15min":0.154},"max":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"last":0.65700000000000003}]},{"osd":1,"last update":"Fri Sep 13 15:43:22 2024","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.42899999999999999}]},{"osd":3,"last update":"Fri Sep 13 15:43:22 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.378}]}]},{"osd":1,"up_from":9,"seq":38654705682,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302260,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069580,"statfs":{"total":96636764160,"available":96327249920,"internally_reserved":0,"allocated":761856,"data_stored":638671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 15:43:22 2024","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Sep 13 15:43:22 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.36599999999999999}]},{"osd":3,"last update":"Fri Sep 13 15:43:22 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.35499999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705683,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302256,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069584,"statfs":{"total":96636764160,"available":96327254016,"internally_reserved":0,"allocated":757760,"data_stored":638652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 15:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.307},{"interface":"front","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.32100000000000001}]},{"osd":2,"last update":"Fri Sep 13 15:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.35899999999999999}]},{"osd":3,"last update":"Fri Sep 13 15:43:25 2024","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.34100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T15:43:45.135 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T15:43:45.135 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T15:43:45.135 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T15:43:45.135 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T15:43:45.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.229+0000 7f420b044640 1 Processor -- start 2024-09-13T15:43:45.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.229+0000 7f420b044640 1 -- start start 2024-09-13T15:43:45.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.229+0000 7f420b044640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f42040953e0 0x7f42040957c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:45.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.229+0000 7f420b044640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f420406d190 0x7f4204132440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:45.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.229+0000 7f420b044640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4204097d40 con 0x7f420406ddf0 2024-09-13T15:43:45.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.229+0000 7f420b044640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f420406f110 con 0x7f420406d190 2024-09-13T15:43:45.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.229+0000 7f420b044640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f420406eb70 con 0x7f42040953e0 2024-09-13T15:43:45.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.230+0000 7f4208de0640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f42040953e0 0x7f42040957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:45.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.230+0000 7f42095e1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f420406d190 0x7f4204132440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:45.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.230+0000 7f42095e1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f420406d190 0x7f4204132440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57920/0 (socket says 172.21.15.70:57920) 2024-09-13T15:43:45.231 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.230+0000 7f4208de0640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f42040953e0 0x7f42040957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:36796/0 (socket says 172.21.15.70:36796) 2024-09-13T15:43:45.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.230+0000 7f4203fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f420406ddf0 0x7f420406ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33648/0 (socket says 172.21.15.70:33648) 2024-09-13T15:43:45.232 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.230+0000 7f42095e1640 1 -- 172.21.15.70:0/2873088778 learned_addr learned my addr 172.21.15.70:0/2873088778 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:45.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.230+0000 7f42095e1640 1 -- 172.21.15.70:0/2873088778 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f42040953e0 msgr2=0x7f42040957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:45.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.230+0000 7f42095e1640 1 --2- 172.21.15.70:0/2873088778 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f42040953e0 0x7f42040957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.230+0000 7f42095e1640 1 -- 172.21.15.70:0/2873088778 >> v1:172.21.15.70:6789/0 conn(0x7f420406ddf0 legacy=0x7f420406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.230+0000 7f42095e1640 1 -- 172.21.15.70:0/2873088778 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4204095d00 con 0x7f420406d190 2024-09-13T15:43:45.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.230+0000 7f4208de0640 1 --2- 172.21.15.70:0/2873088778 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f42040953e0 0x7f42040957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:45.233 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.231+0000 7f42095e1640 1 --2- 172.21.15.70:0/2873088778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f420406d190 0x7f4204132440 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00d150 tx=0x7f41fc02c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f7d115dfa10985 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.231+0000 7f42037fe640 1 -- 172.21.15.70:0/2873088778 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc009600 con 0x7f420406d190 2024-09-13T15:43:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.231+0000 7f42037fe640 1 -- 172.21.15.70:0/2873088778 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41fc030050 con 0x7f420406d190 2024-09-13T15:43:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.231+0000 7f42037fe640 1 -- 172.21.15.70:0/2873088778 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc0099a0 con 0x7f420406d190 2024-09-13T15:43:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.231+0000 7f420b044640 1 -- 172.21.15.70:0/2873088778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f420406d190 msgr2=0x7f4204132440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.231+0000 7f420b044640 1 --2- 172.21.15.70:0/2873088778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f420406d190 0x7f4204132440 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00d150 tx=0x7f41fc02c810 comp rx=0 tx=0).stop 2024-09-13T15:43:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.231+0000 7f420b044640 1 -- 172.21.15.70:0/2873088778 shutdown_connections 2024-09-13T15:43:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.231+0000 7f420b044640 1 --2- 172.21.15.70:0/2873088778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f420406d190 0x7f4204132440 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.231+0000 7f420b044640 1 --2- 172.21.15.70:0/2873088778 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f42040953e0 0x7f42040957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.231+0000 7f420b044640 1 -- 172.21.15.70:0/2873088778 >> 172.21.15.70:0/2873088778 conn(0x7f4204098b10 msgr2=0x7f42040963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.231+0000 7f420b044640 1 -- 172.21.15.70:0/2873088778 shutdown_connections 2024-09-13T15:43:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.231+0000 7f420b044640 1 -- 172.21.15.70:0/2873088778 wait complete. 2024-09-13T15:43:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.232+0000 7f420b044640 1 Processor -- start 2024-09-13T15:43:45.234 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.232+0000 7f420b044640 1 -- start start 2024-09-13T15:43:45.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f420b044640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f420406d190 0x7f42041be200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:45.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f420b044640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f420406ddf0 0x7f42041be740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:45.236 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f420b044640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42040953e0 0x7f42041228f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:45.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f420b044640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4204124b70 con 0x7f420406ddf0 2024-09-13T15:43:45.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f420b044640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420408c5d0 con 0x7f42040953e0 2024-09-13T15:43:45.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f420b044640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f420408c300 con 0x7f420406d190 2024-09-13T15:43:45.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f4208de0640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f420406d190 0x7f42041be200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:45.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f4203fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f420406ddf0 0x7f42041be740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:45.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f4208de0640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f420406d190 0x7f42041be200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:36804/0 (socket says 172.21.15.70:36804) 2024-09-13T15:43:45.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f4208de0640 1 -- 172.21.15.70:0/1225546056 learned_addr learned my addr 172.21.15.70:0/1225546056 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:45.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f42095e1640 1 --2- 172.21.15.70:0/1225546056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42040953e0 0x7f42041228f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:45.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f4203fff640 1 -- 172.21.15.70:0/1225546056 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f420406d190 msgr2=0x7f42041be200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f4203fff640 1 --2- 172.21.15.70:0/1225546056 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f420406d190 0x7f42041be200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f4203fff640 1 -- 172.21.15.70:0/1225546056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42040953e0 msgr2=0x7f42041228f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f4203fff640 1 --2- 172.21.15.70:0/1225546056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42040953e0 0x7f42041228f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.237 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f4203fff640 1 -- 172.21.15.70:0/1225546056 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f8009400 con 0x7f420406ddf0 2024-09-13T15:43:45.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f4203fff640 1 --2- 172.21.15.70:0/1225546056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f420406ddf0 0x7f42041be740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f41f000c430 tx=0x7f41f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:45.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f42017fa640 1 -- 172.21.15.70:0/1225546056 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f00074e0 con 0x7f420406ddf0 2024-09-13T15:43:45.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.233+0000 7f42017fa640 1 -- 172.21.15.70:0/1225546056 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f0007640 con 0x7f420406ddf0 2024-09-13T15:43:45.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.234+0000 7f42017fa640 1 -- 172.21.15.70:0/1225546056 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0004a00 con 0x7f420406ddf0 2024-09-13T15:43:45.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.234+0000 7f420b044640 1 -- 172.21.15.70:0/1225546056 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41fc00bd40 con 0x7f420406ddf0 2024-09-13T15:43:45.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.234+0000 7f420b044640 1 -- 172.21.15.70:0/1225546056 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420408b9b0 con 0x7f420406ddf0 2024-09-13T15:43:45.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.234+0000 7f42017fa640 1 -- 172.21.15.70:0/1225546056 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 9) v1 ==== 55156+0+0 (secure 0 0 0) 0x7f41f0004b60 con 0x7f420406ddf0 2024-09-13T15:43:45.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.234+0000 7f420b044640 1 -- 172.21.15.70:0/1225546056 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42040957c0 con 0x7f420406ddf0 2024-09-13T15:43:45.238 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.235+0000 7f42017fa640 1 -- 172.21.15.70:0/1225546056 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f41f005c170 con 0x7f420406ddf0 2024-09-13T15:43:45.239 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.238+0000 7f42017fa640 1 -- 172.21.15.70:0/1225546056 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f41f0039e60 con 0x7f420406ddf0 2024-09-13T15:43:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.393+0000 7f420b044640 1 -- 172.21.15.70:0/1225546056 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4204066fa0 con 0x7f420406ddf0 2024-09-13T15:43:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.393+0000 7f42017fa640 1 -- 172.21.15.70:0/1225546056 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f41f003d480 con 0x7f420406ddf0 2024-09-13T15:43:45.394 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:43:45.395 INFO:teuthology.orchestra.run.smithi070.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T15:43:45.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.395+0000 7f420b044640 1 -- 172.21.15.70:0/1225546056 >> v1:172.21.15.70:6816/1229510400 conn(0x7f41e80474d0 legacy=0x7f41e8049990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.395+0000 7f420b044640 1 -- 172.21.15.70:0/1225546056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f420406ddf0 msgr2=0x7f42041be740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.395+0000 7f420b044640 1 --2- 172.21.15.70:0/1225546056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f420406ddf0 0x7f42041be740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f41f000c430 tx=0x7f41f000c8f0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.395+0000 7f420b044640 1 -- 172.21.15.70:0/1225546056 shutdown_connections 2024-09-13T15:43:45.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.395+0000 7f420b044640 1 --2- 172.21.15.70:0/1225546056 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f42040953e0 0x7f42041228f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.395+0000 7f420b044640 1 --2- 172.21.15.70:0/1225546056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f420406ddf0 0x7f42041be740 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.395+0000 7f420b044640 1 --2- 172.21.15.70:0/1225546056 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f420406d190 0x7f42041be200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.396 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.395+0000 7f420b044640 1 -- 172.21.15.70:0/1225546056 >> 172.21.15.70:0/1225546056 conn(0x7f4204098b10 msgr2=0x7f4204126cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:45.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.395+0000 7f420b044640 1 -- 172.21.15.70:0/1225546056 shutdown_connections 2024-09-13T15:43:45.397 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.395+0000 7f420b044640 1 -- 172.21.15.70:0/1225546056 wait complete. 2024-09-13T15:43:45.406 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T15:43:45.406 INFO:teuthology.run_tasks:Running task ceph.restart... 2024-09-13T15:43:45.417 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T15:43:45.417 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:43:45.445 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T15:43:45.445 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-13T15:43:45.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.585+0000 7fa6dc7b5640 1 Processor -- start 2024-09-13T15:43:45.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.585+0000 7fa6dc7b5640 1 -- start start 2024-09-13T15:43:45.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.586+0000 7fa6dc7b5640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d4126f00 0x7fa6d41272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:45.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.586+0000 7fa6dc7b5640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa6d4127820 0x7fa6d406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.586+0000 7fa6dc7b5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d4126b40 con 0x7fa6d406cfc0 2024-09-13T15:43:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.586+0000 7fa6dc7b5640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d406e310 con 0x7fa6d4126f00 2024-09-13T15:43:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.586+0000 7fa6dc7b5640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d406e850 con 0x7fa6d4127820 2024-09-13T15:43:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.586+0000 7fa6da551640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d4126f00 0x7fa6d41272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.586+0000 7fa6d9d50640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa6d4127820 0x7fa6d406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.586+0000 7fa6dad52640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa6d406cfc0 0x7fa6d41313b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33656/0 (socket says 172.21.15.70:33656) 2024-09-13T15:43:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.586+0000 7fa6dad52640 1 -- 172.21.15.70:0/4213152791 learned_addr learned my addr 172.21.15.70:0/4213152791 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.586+0000 7fa6d954f640 1 -- 172.21.15.70:0/4213152791 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123484132 0 0) 0x7fa6d4126b40 con 0x7fa6d406cfc0 2024-09-13T15:43:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.586+0000 7fa6d954f640 1 -- 172.21.15.70:0/4213152791 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6bc003520 con 0x7fa6d406cfc0 2024-09-13T15:43:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.586+0000 7fa6d9d50640 1 -- 172.21.15.70:0/4213152791 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d4126f00 msgr2=0x7fa6d41272e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.586+0000 7fa6d9d50640 1 --2- 172.21.15.70:0/4213152791 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d4126f00 0x7fa6d41272e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.587+0000 7fa6d9d50640 1 -- 172.21.15.70:0/4213152791 >> v1:172.21.15.70:6789/0 conn(0x7fa6d406cfc0 legacy=0x7fa6d41313b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.587+0000 7fa6d9d50640 1 -- 172.21.15.70:0/4213152791 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d406f030 con 0x7fa6d4127820 2024-09-13T15:43:45.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.587+0000 7fa6da551640 1 --2- 172.21.15.70:0/4213152791 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d4126f00 0x7fa6d41272e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:45.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.587+0000 7fa6d9d50640 1 --2- 172.21.15.70:0/4213152791 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa6d4127820 0x7fa6d406ca80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80097d0 tx=0x7fa6c802a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ff78533a6b6e1f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:45.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.587+0000 7fa6d954f640 1 -- 172.21.15.70:0/4213152791 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c802ad90 con 0x7fa6d4127820 2024-09-13T15:43:45.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.587+0000 7fa6d954f640 1 -- 172.21.15.70:0/4213152791 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c8032440 con 0x7fa6d4127820 2024-09-13T15:43:45.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.587+0000 7fa6d954f640 1 -- 172.21.15.70:0/4213152791 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c80327b0 con 0x7fa6d4127820 2024-09-13T15:43:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.588+0000 7fa6dc7b5640 1 -- 172.21.15.70:0/4213152791 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa6d4127820 msgr2=0x7fa6d406ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.588+0000 7fa6dc7b5640 1 --2- 172.21.15.70:0/4213152791 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa6d4127820 0x7fa6d406ca80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80097d0 tx=0x7fa6c802a040 comp rx=0 tx=0).stop 2024-09-13T15:43:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.588+0000 7fa6dc7b5640 1 -- 172.21.15.70:0/4213152791 shutdown_connections 2024-09-13T15:43:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.588+0000 7fa6dc7b5640 1 --2- 172.21.15.70:0/4213152791 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa6d4127820 0x7fa6d406ca80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.588+0000 7fa6dc7b5640 1 --2- 172.21.15.70:0/4213152791 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d4126f00 0x7fa6d41272e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.588+0000 7fa6dc7b5640 1 -- 172.21.15.70:0/4213152791 >> 172.21.15.70:0/4213152791 conn(0x7fa6d4098ae0 msgr2=0x7fa6d40963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.588+0000 7fa6dc7b5640 1 -- 172.21.15.70:0/4213152791 shutdown_connections 2024-09-13T15:43:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.588+0000 7fa6dc7b5640 1 -- 172.21.15.70:0/4213152791 wait complete. 2024-09-13T15:43:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.589+0000 7fa6dc7b5640 1 Processor -- start 2024-09-13T15:43:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.589+0000 7fa6dc7b5640 1 -- start start 2024-09-13T15:43:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.589+0000 7fa6dc7b5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d406cfc0 0x7fa6d41c0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.589+0000 7fa6dc7b5640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d4126f00 0x7fa6d41c0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.589+0000 7fa6dc7b5640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa6d4127820 0x7fa6d41c0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.589+0000 7fa6dc7b5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d4071cc0 con 0x7fa6d406cfc0 2024-09-13T15:43:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.589+0000 7fa6dc7b5640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d408f5e0 con 0x7fa6d4126f00 2024-09-13T15:43:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.589+0000 7fa6dc7b5640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d408f830 con 0x7fa6d4127820 2024-09-13T15:43:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.589+0000 7fa6da551640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d406cfc0 0x7fa6d41c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.589+0000 7fa6da551640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d406cfc0 0x7fa6d41c0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57942/0 (socket says 172.21.15.70:57942) 2024-09-13T15:43:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.589+0000 7fa6d9d50640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d4126f00 0x7fa6d41c0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.589+0000 7fa6da551640 1 -- 172.21.15.70:0/167583618 learned_addr learned my addr 172.21.15.70:0/167583618 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.589+0000 7fa6dad52640 1 --2- 172.21.15.70:0/167583618 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa6d4127820 0x7fa6d41c0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.590+0000 7fa6da551640 1 -- 172.21.15.70:0/167583618 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa6d4127820 msgr2=0x7fa6d41c0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.590+0000 7fa6da551640 1 --2- 172.21.15.70:0/167583618 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa6d4127820 0x7fa6d41c0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.590+0000 7fa6da551640 1 -- 172.21.15.70:0/167583618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d4126f00 msgr2=0x7fa6d41c0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.590+0000 7fa6da551640 1 --2- 172.21.15.70:0/167583618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d4126f00 0x7fa6d41c0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.590+0000 7fa6da551640 1 -- 172.21.15.70:0/167583618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6cc009400 con 0x7fa6d406cfc0 2024-09-13T15:43:45.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.590+0000 7fa6da551640 1 --2- 172.21.15.70:0/167583618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d406cfc0 0x7fa6d41c0110 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00ebe0 tx=0x7fa6cc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:45.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.590+0000 7fa6d2ffd640 1 -- 172.21.15.70:0/167583618 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc009760 con 0x7fa6d406cfc0 2024-09-13T15:43:45.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.590+0000 7fa6d2ffd640 1 -- 172.21.15.70:0/167583618 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc00cc30 con 0x7fa6d406cfc0 2024-09-13T15:43:45.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.590+0000 7fa6dc7b5640 1 -- 172.21.15.70:0/167583618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c8009400 con 0x7fa6d406cfc0 2024-09-13T15:43:45.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.590+0000 7fa6d2ffd640 1 -- 172.21.15.70:0/167583618 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00eee0 con 0x7fa6d406cfc0 2024-09-13T15:43:45.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.590+0000 7fa6dc7b5640 1 -- 172.21.15.70:0/167583618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d406c3a0 con 0x7fa6d406cfc0 2024-09-13T15:43:45.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.591+0000 7fa6d2ffd640 1 -- 172.21.15.70:0/167583618 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fa6cc013470 con 0x7fa6d406cfc0 2024-09-13T15:43:45.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.591+0000 7fa6dc7b5640 1 -- 172.21.15.70:0/167583618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d41272e0 con 0x7fa6d406cfc0 2024-09-13T15:43:45.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.591+0000 7fa6d2ffd640 1 -- 172.21.15.70:0/167583618 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fa6cc05cc60 con 0x7fa6d406cfc0 2024-09-13T15:43:45.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.595+0000 7fa6d2ffd640 1 -- 172.21.15.70:0/167583618 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fa6cc03a880 con 0x7fa6d406cfc0 2024-09-13T15:43:45.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.740+0000 7fa6dc7b5640 1 -- 172.21.15.70:0/167583618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd down", "ids": ["0"]} v 0) v1 -- 0x7fa6d4066fa0 con 0x7fa6d406cfc0 2024-09-13T15:43:45.741 INFO:tasks.ceph.mon.a.smithi070.stderr:2024-09-13T15:43:45.740+0000 7fee39368640 -1 mon.a@0(leader).osd e20 definitely_dead 0 2024-09-13T15:43:45.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.740+0000 7fa6d2ffd640 1 -- 172.21.15.70:0/167583618 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["0"]}]=0 osd.0 is already down. v20) v1 ==== 93+0+0 (secure 0 0 0) 0x7fa6cc021030 con 0x7fa6d406cfc0 2024-09-13T15:43:45.742 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0 is already down. 2024-09-13T15:43:45.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.742+0000 7fa6dc7b5640 1 -- 172.21.15.70:0/167583618 >> v1:172.21.15.70:6816/1229510400 conn(0x7fa6c0047560 legacy=0x7fa6c0049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.742+0000 7fa6dc7b5640 1 -- 172.21.15.70:0/167583618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d406cfc0 msgr2=0x7fa6d41c0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.742+0000 7fa6dc7b5640 1 --2- 172.21.15.70:0/167583618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d406cfc0 0x7fa6d41c0110 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00ebe0 tx=0x7fa6cc00c520 comp rx=0 tx=0).stop 2024-09-13T15:43:45.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.742+0000 7fa6dc7b5640 1 -- 172.21.15.70:0/167583618 shutdown_connections 2024-09-13T15:43:45.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.742+0000 7fa6dc7b5640 1 --2- 172.21.15.70:0/167583618 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa6d4127820 0x7fa6d41c0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.742+0000 7fa6dc7b5640 1 --2- 172.21.15.70:0/167583618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa6d4126f00 0x7fa6d41c0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.742+0000 7fa6dc7b5640 1 --2- 172.21.15.70:0/167583618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d406cfc0 0x7fa6d41c0110 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.742+0000 7fa6dc7b5640 1 -- 172.21.15.70:0/167583618 >> 172.21.15.70:0/167583618 conn(0x7fa6d4098ae0 msgr2=0x7fa6d406fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:45.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.742+0000 7fa6dc7b5640 1 -- 172.21.15.70:0/167583618 shutdown_connections 2024-09-13T15:43:45.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.742+0000 7fa6dc7b5640 1 -- 172.21.15.70:0/167583618 wait complete. 2024-09-13T15:43:45.754 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T15:43:45.754 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T15:43:45.756 INFO:tasks.ceph.osd.0:Started 2024-09-13T15:43:45.756 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T15:43:45.756 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:43:45.804 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T15:43:45.804 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-09-13T15:43:45.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.901+0000 7fea4eef0640 1 Processor -- start 2024-09-13T15:43:45.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.901+0000 7fea4eef0640 1 -- start start 2024-09-13T15:43:45.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.902+0000 7fea4eef0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea4806ca80 0x7fea4806ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:45.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.902+0000 7fea4eef0640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea4806dd10 0x7fea4806d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:45.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.902+0000 7fea4eef0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4806d8e0 con 0x7fea48095090 2024-09-13T15:43:45.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.902+0000 7fea4eef0640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4806f1d0 con 0x7fea4806ca80 2024-09-13T15:43:45.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.902+0000 7fea4eef0640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea4806ebc0 con 0x7fea4806dd10 2024-09-13T15:43:45.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.902+0000 7fea47fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea4806dd10 0x7fea4806d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:45.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.902+0000 7fea4cc8c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea4806ca80 0x7fea4806ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:45.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.902+0000 7fea4d48d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fea48095090 0x7fea48095540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33674/0 (socket says 172.21.15.70:33674) 2024-09-13T15:43:45.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.902+0000 7fea4d48d640 1 -- 172.21.15.70:0/4147873754 learned_addr learned my addr 172.21.15.70:0/4147873754 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:45.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.902+0000 7fea4cc8c640 1 -- 172.21.15.70:0/4147873754 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea4806dd10 msgr2=0x7fea4806d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:45.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.902+0000 7fea4cc8c640 1 --2- 172.21.15.70:0/4147873754 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea4806dd10 0x7fea4806d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.902+0000 7fea4cc8c640 1 -- 172.21.15.70:0/4147873754 >> v1:172.21.15.70:6789/0 conn(0x7fea48095090 legacy=0x7fea48095540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.902+0000 7fea477fe640 1 -- 172.21.15.70:0/4147873754 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646143612 0 0) 0x7fea4806d8e0 con 0x7fea48095090 2024-09-13T15:43:45.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.902+0000 7fea4cc8c640 1 -- 172.21.15.70:0/4147873754 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea48097d40 con 0x7fea4806ca80 2024-09-13T15:43:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.902+0000 7fea47fff640 1 --2- 172.21.15.70:0/4147873754 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea4806dd10 0x7fea4806d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.903+0000 7fea4cc8c640 1 --2- 172.21.15.70:0/4147873754 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea4806ca80 0x7fea4806ce60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fea3800dbf0 tx=0x7fea3802e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=739719633bad0013 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.903+0000 7fea477fe640 1 -- 172.21.15.70:0/4147873754 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3802eea0 con 0x7fea4806ca80 2024-09-13T15:43:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.903+0000 7fea477fe640 1 -- 172.21.15.70:0/4147873754 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3800d2d0 con 0x7fea4806ca80 2024-09-13T15:43:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.903+0000 7fea477fe640 1 -- 172.21.15.70:0/4147873754 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3800b670 con 0x7fea4806ca80 2024-09-13T15:43:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.903+0000 7fea4eef0640 1 -- 172.21.15.70:0/4147873754 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea4806ca80 msgr2=0x7fea4806ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.903+0000 7fea4eef0640 1 --2- 172.21.15.70:0/4147873754 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea4806ca80 0x7fea4806ce60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fea3800dbf0 tx=0x7fea3802e350 comp rx=0 tx=0).stop 2024-09-13T15:43:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.903+0000 7fea4eef0640 1 -- 172.21.15.70:0/4147873754 shutdown_connections 2024-09-13T15:43:45.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.903+0000 7fea4eef0640 1 --2- 172.21.15.70:0/4147873754 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea4806dd10 0x7fea4806d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.903+0000 7fea4eef0640 1 --2- 172.21.15.70:0/4147873754 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea4806ca80 0x7fea4806ce60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.903+0000 7fea4eef0640 1 -- 172.21.15.70:0/4147873754 >> 172.21.15.70:0/4147873754 conn(0x7fea48098ae0 msgr2=0x7fea480963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:45.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.903+0000 7fea4eef0640 1 -- 172.21.15.70:0/4147873754 shutdown_connections 2024-09-13T15:43:45.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.903+0000 7fea4eef0640 1 -- 172.21.15.70:0/4147873754 wait complete. 2024-09-13T15:43:45.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.904+0000 7fea4eef0640 1 Processor -- start 2024-09-13T15:43:45.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.904+0000 7fea4eef0640 1 -- start start 2024-09-13T15:43:45.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea4eef0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea4806ca80 0x7fea48125730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:45.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea4eef0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea4806dd10 0x7fea48125c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:45.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea4eef0640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea48095090 0x7fea481261b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:45.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea4eef0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea48126ff0 con 0x7fea4806ca80 2024-09-13T15:43:45.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea4eef0640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea481266f0 con 0x7fea4806dd10 2024-09-13T15:43:45.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea4eef0640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea481277f0 con 0x7fea48095090 2024-09-13T15:43:45.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea47fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea4806dd10 0x7fea48125c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:45.911 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea4cc8c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea4806ca80 0x7fea48125730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea4d48d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea48095090 0x7fea481261b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea47fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea4806dd10 0x7fea48125c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:40640/0 (socket says 172.21.15.70:40640) 2024-09-13T15:43:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea4cc8c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea4806ca80 0x7fea48125730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57958/0 (socket says 172.21.15.70:57958) 2024-09-13T15:43:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea47fff640 1 -- 172.21.15.70:0/2680549491 learned_addr learned my addr 172.21.15.70:0/2680549491 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea4d48d640 1 -- 172.21.15.70:0/2680549491 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea4806dd10 msgr2=0x7fea48125c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea4d48d640 1 --2- 172.21.15.70:0/2680549491 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea4806dd10 0x7fea48125c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea4d48d640 1 -- 172.21.15.70:0/2680549491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea4806ca80 msgr2=0x7fea48125730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea4d48d640 1 --2- 172.21.15.70:0/2680549491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea4806ca80 0x7fea48125730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea4d48d640 1 -- 172.21.15.70:0/2680549491 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c00b4b0 con 0x7fea48095090 2024-09-13T15:43:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea47fff640 1 --2- 172.21.15.70:0/2680549491 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea4806dd10 0x7fea48125c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea4d48d640 1 --2- 172.21.15.70:0/2680549491 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea48095090 0x7fea481261b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fea40001560 tx=0x7fea4000cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.905+0000 7fea457fa640 1 -- 172.21.15.70:0/2680549491 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4000ce30 con 0x7fea48095090 2024-09-13T15:43:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.906+0000 7fea457fa640 1 -- 172.21.15.70:0/2680549491 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4000a640 con 0x7fea48095090 2024-09-13T15:43:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.906+0000 7fea4eef0640 1 -- 172.21.15.70:0/2680549491 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3800d8a0 con 0x7fea48095090 2024-09-13T15:43:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.906+0000 7fea457fa640 1 -- 172.21.15.70:0/2680549491 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4000aa10 con 0x7fea48095090 2024-09-13T15:43:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.906+0000 7fea4eef0640 1 -- 172.21.15.70:0/2680549491 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4808b760 con 0x7fea48095090 2024-09-13T15:43:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.906+0000 7fea4eef0640 1 -- 172.21.15.70:0/2680549491 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4809a200 con 0x7fea48095090 2024-09-13T15:43:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.906+0000 7fea457fa640 1 -- 172.21.15.70:0/2680549491 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fea4000ab70 con 0x7fea48095090 2024-09-13T15:43:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.907+0000 7fea457fa640 1 -- 172.21.15.70:0/2680549491 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fea4005c020 con 0x7fea48095090 2024-09-13T15:43:45.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:45.910+0000 7fea457fa640 1 -- 172.21.15.70:0/2680549491 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fea40039ca0 con 0x7fea48095090 2024-09-13T15:43:46.054 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.053+0000 7fea4eef0640 1 -- 172.21.15.70:0/2680549491 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd down", "ids": ["1"]} v 0) v1 -- 0x7fea48095540 con 0x7fea48095090 2024-09-13T15:43:46.054 INFO:tasks.ceph.mon.a.smithi070.stderr:2024-09-13T15:43:46.053+0000 7fee39368640 -1 mon.a@0(leader).osd e20 definitely_dead 0 2024-09-13T15:43:46.341 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-13T15:43:46.339+0000 7f9e56da4640 -1 Falling back to public interface 2024-09-13T15:43:46.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.545+0000 7fea457fa640 1 -- 172.21.15.70:0/2680549491 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["1"]}]=0 marked down osd.1. v21) v1 ==== 89+0+0 (secure 0 0 0) 0x7fea40018030 con 0x7fea48095090 2024-09-13T15:43:46.546 INFO:teuthology.orchestra.run.smithi070.stderr:marked down osd.1. 2024-09-13T15:43:46.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.547+0000 7fea4eef0640 1 -- 172.21.15.70:0/2680549491 >> v1:172.21.15.70:6816/1229510400 conn(0x7fea34047560 legacy=0x7fea34049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:46.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.547+0000 7fea4eef0640 1 -- 172.21.15.70:0/2680549491 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea48095090 msgr2=0x7fea481261b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:46.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.547+0000 7fea4eef0640 1 --2- 172.21.15.70:0/2680549491 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea48095090 0x7fea481261b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fea40001560 tx=0x7fea4000cbe0 comp rx=0 tx=0).stop 2024-09-13T15:43:46.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.547+0000 7fea4eef0640 1 -- 172.21.15.70:0/2680549491 shutdown_connections 2024-09-13T15:43:46.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.547+0000 7fea4eef0640 1 --2- 172.21.15.70:0/2680549491 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fea48095090 0x7fea481261b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:46.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.547+0000 7fea4eef0640 1 --2- 172.21.15.70:0/2680549491 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea4806dd10 0x7fea48125c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:46.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.547+0000 7fea4eef0640 1 --2- 172.21.15.70:0/2680549491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea4806ca80 0x7fea48125730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:46.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.548+0000 7fea4eef0640 1 -- 172.21.15.70:0/2680549491 >> 172.21.15.70:0/2680549491 conn(0x7fea48098ae0 msgr2=0x7fea48096fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:46.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.548+0000 7fea4eef0640 1 -- 172.21.15.70:0/2680549491 shutdown_connections 2024-09-13T15:43:46.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.548+0000 7fea4eef0640 1 -- 172.21.15.70:0/2680549491 wait complete. 2024-09-13T15:43:46.560 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T15:43:46.560 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T15:43:46.562 INFO:tasks.ceph.osd.1:Started 2024-09-13T15:43:46.563 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T15:43:46.563 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:43:46.601 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T15:43:46.601 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-09-13T15:43:46.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.739+0000 7fb33e227640 1 Processor -- start 2024-09-13T15:43:46.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.739+0000 7fb33e227640 1 -- start start 2024-09-13T15:43:46.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.739+0000 7fb33e227640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb33806c980 0x7fb33812ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:46.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.739+0000 7fb33e227640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb33812f280 0x7fb338131670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.740+0000 7fb33e227640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33806e7e0 con 0x7fb338129d70 2024-09-13T15:43:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.740+0000 7fb33e227640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33806ebb0 con 0x7fb33812f280 2024-09-13T15:43:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.740+0000 7fb33e227640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb33812a860 con 0x7fb33806c980 2024-09-13T15:43:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.740+0000 7fb336ffd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb33806c980 0x7fb33812ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.740+0000 7fb337fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb33812f280 0x7fb338131670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.740+0000 7fb3377fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb338129d70 0x7fb33812a150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33680/0 (socket says 172.21.15.70:33680) 2024-09-13T15:43:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.740+0000 7fb3377fe640 1 -- 172.21.15.70:0/2731378140 learned_addr learned my addr 172.21.15.70:0/2731378140 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.740+0000 7fb337fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb33812f280 0x7fb338131670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57980/0 (socket says 172.21.15.70:57980) 2024-09-13T15:43:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.740+0000 7fb336ffd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb33806c980 0x7fb33812ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:36838/0 (socket says 172.21.15.70:36838) 2024-09-13T15:43:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.740+0000 7fb336ffd640 1 -- 172.21.15.70:0/2731378140 >> v2:172.21.15.70:3300/0 conn(0x7fb33812f280 msgr2=0x7fb338131670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.740+0000 7fb336ffd640 1 --2- 172.21.15.70:0/2731378140 >> v2:172.21.15.70:3300/0 conn(0x7fb33812f280 0x7fb338131670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.740+0000 7fb336ffd640 1 -- 172.21.15.70:0/2731378140 >> v1:172.21.15.70:6789/0 conn(0x7fb338129d70 legacy=0x7fb33812a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.740+0000 7fb336ffd640 1 -- 172.21.15.70:0/2731378140 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33806e000 con 0x7fb33806c980 2024-09-13T15:43:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.740+0000 7fb3367fc640 1 -- 172.21.15.70:0/2731378140 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360967557 0 0) 0x7fb33806e7e0 con 0x7fb338129d70 2024-09-13T15:43:46.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.740+0000 7fb337fff640 1 --2- 172.21.15.70:0/2731378140 >> v2:172.21.15.70:3300/0 conn(0x7fb33812f280 0x7fb338131670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:46.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.741+0000 7fb336ffd640 1 --2- 172.21.15.70:0/2731378140 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb33806c980 0x7fb33812ed40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00a940 tx=0x7fb32c02b430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7473278d16885573 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:46.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.741+0000 7fb3367fc640 1 -- 172.21.15.70:0/2731378140 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c030710 con 0x7fb33806c980 2024-09-13T15:43:46.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.741+0000 7fb3367fc640 1 -- 172.21.15.70:0/2731378140 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c02e050 con 0x7fb33806c980 2024-09-13T15:43:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.741+0000 7fb3367fc640 1 -- 172.21.15.70:0/2731378140 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c030a90 con 0x7fb33806c980 2024-09-13T15:43:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.741+0000 7fb33e227640 1 -- 172.21.15.70:0/2731378140 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb33806c980 msgr2=0x7fb33812ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.741+0000 7fb33e227640 1 --2- 172.21.15.70:0/2731378140 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb33806c980 0x7fb33812ed40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00a940 tx=0x7fb32c02b430 comp rx=0 tx=0).stop 2024-09-13T15:43:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.741+0000 7fb33e227640 1 -- 172.21.15.70:0/2731378140 shutdown_connections 2024-09-13T15:43:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.741+0000 7fb33e227640 1 --2- 172.21.15.70:0/2731378140 >> v2:172.21.15.70:3300/0 conn(0x7fb33812f280 0x7fb338131670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.741+0000 7fb33e227640 1 --2- 172.21.15.70:0/2731378140 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb33806c980 0x7fb33812ed40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.741+0000 7fb33e227640 1 -- 172.21.15.70:0/2731378140 >> 172.21.15.70:0/2731378140 conn(0x7fb338098ae0 msgr2=0x7fb3380963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.741+0000 7fb33e227640 1 -- 172.21.15.70:0/2731378140 shutdown_connections 2024-09-13T15:43:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.741+0000 7fb33e227640 1 -- 172.21.15.70:0/2731378140 wait complete. 2024-09-13T15:43:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.742+0000 7fb33e227640 1 Processor -- start 2024-09-13T15:43:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.742+0000 7fb33e227640 1 -- start start 2024-09-13T15:43:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb33e227640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb33806c980 0x7fb3381c2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb33e227640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb338129d70 0x7fb3381c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb33e227640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb33812f280 0x7fb3381c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb33e227640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb338095b60 con 0x7fb33812f280 2024-09-13T15:43:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb33e227640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33808c420 con 0x7fb33806c980 2024-09-13T15:43:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb33e227640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb33808d700 con 0x7fb338129d70 2024-09-13T15:43:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb336ffd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb338129d70 0x7fb3381c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb3377fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb33806c980 0x7fb3381c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb337fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb33812f280 0x7fb3381c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb336ffd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb338129d70 0x7fb3381c2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:36852/0 (socket says 172.21.15.70:36852) 2024-09-13T15:43:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb3377fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb33806c980 0x7fb3381c2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:40670/0 (socket says 172.21.15.70:40670) 2024-09-13T15:43:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb336ffd640 1 -- 172.21.15.70:0/3100766044 learned_addr learned my addr 172.21.15.70:0/3100766044 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb337fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb33812f280 0x7fb3381c2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57990/0 (socket says 172.21.15.70:57990) 2024-09-13T15:43:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb337fff640 1 -- 172.21.15.70:0/3100766044 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb338129d70 msgr2=0x7fb3381c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb337fff640 1 --2- 172.21.15.70:0/3100766044 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb338129d70 0x7fb3381c2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb337fff640 1 -- 172.21.15.70:0/3100766044 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb33806c980 msgr2=0x7fb3381c2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb337fff640 1 --2- 172.21.15.70:0/3100766044 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb33806c980 0x7fb3381c2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb337fff640 1 -- 172.21.15.70:0/3100766044 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb330009400 con 0x7fb33812f280 2024-09-13T15:43:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb337fff640 1 --2- 172.21.15.70:0/3100766044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb33812f280 0x7fb3381c2fb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb33000e850 tx=0x7fb33000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb31ffff640 1 -- 172.21.15.70:0/3100766044 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330009650 con 0x7fb33812f280 2024-09-13T15:43:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb31ffff640 1 -- 172.21.15.70:0/3100766044 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb330004020 con 0x7fb33812f280 2024-09-13T15:43:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb33e227640 1 -- 172.21.15.70:0/3100766044 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32c00a5a0 con 0x7fb33812f280 2024-09-13T15:43:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.743+0000 7fb31ffff640 1 -- 172.21.15.70:0/3100766044 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3300043d0 con 0x7fb33812f280 2024-09-13T15:43:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.744+0000 7fb33e227640 1 -- 172.21.15.70:0/3100766044 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33808cc70 con 0x7fb33812f280 2024-09-13T15:43:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.744+0000 7fb31ffff640 1 -- 172.21.15.70:0/3100766044 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fb33000c8e0 con 0x7fb33812f280 2024-09-13T15:43:46.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.744+0000 7fb33e227640 1 -- 172.21.15.70:0/3100766044 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33809a7e0 con 0x7fb33812f280 2024-09-13T15:43:46.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.745+0000 7fb31ffff640 1 -- 172.21.15.70:0/3100766044 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fb33005c720 con 0x7fb33812f280 2024-09-13T15:43:46.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.748+0000 7fb31ffff640 1 -- 172.21.15.70:0/3100766044 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fb33003a3a0 con 0x7fb33812f280 2024-09-13T15:43:46.893 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:46.892+0000 7fb33e227640 1 -- 172.21.15.70:0/3100766044 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd down", "ids": ["2"]} v 0) v1 -- 0x7fb33812a150 con 0x7fb33812f280 2024-09-13T15:43:46.894 INFO:tasks.ceph.mon.a.smithi070.stderr:2024-09-13T15:43:46.892+0000 7fee39368640 -1 mon.a@0(leader).osd e21 definitely_dead 0 2024-09-13T15:43:47.156 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-13T15:43:47.155+0000 7fe783eeb640 -1 Falling back to public interface 2024-09-13T15:43:47.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.551+0000 7fb31ffff640 1 -- 172.21.15.70:0/3100766044 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["2"]}]=0 marked down osd.2. v22) v1 ==== 89+0+0 (secure 0 0 0) 0x7fb330017070 con 0x7fb33812f280 2024-09-13T15:43:47.552 INFO:teuthology.orchestra.run.smithi070.stderr:marked down osd.2. 2024-09-13T15:43:47.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.553+0000 7fb33e227640 1 -- 172.21.15.70:0/3100766044 >> v1:172.21.15.70:6816/1229510400 conn(0x7fb324047560 legacy=0x7fb324049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:47.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.553+0000 7fb33e227640 1 -- 172.21.15.70:0/3100766044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb33812f280 msgr2=0x7fb3381c2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:47.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.553+0000 7fb33e227640 1 --2- 172.21.15.70:0/3100766044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb33812f280 0x7fb3381c2fb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb33000e850 tx=0x7fb33000ed10 comp rx=0 tx=0).stop 2024-09-13T15:43:47.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.553+0000 7fb33e227640 1 -- 172.21.15.70:0/3100766044 shutdown_connections 2024-09-13T15:43:47.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.553+0000 7fb33e227640 1 --2- 172.21.15.70:0/3100766044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb33812f280 0x7fb3381c2fb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:47.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.553+0000 7fb33e227640 1 --2- 172.21.15.70:0/3100766044 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb338129d70 0x7fb3381c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:47.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.553+0000 7fb33e227640 1 --2- 172.21.15.70:0/3100766044 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb33806c980 0x7fb3381c2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:47.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.553+0000 7fb33e227640 1 -- 172.21.15.70:0/3100766044 >> 172.21.15.70:0/3100766044 conn(0x7fb338098ae0 msgr2=0x7fb33809a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:47.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.553+0000 7fb33e227640 1 -- 172.21.15.70:0/3100766044 shutdown_connections 2024-09-13T15:43:47.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.553+0000 7fb33e227640 1 -- 172.21.15.70:0/3100766044 wait complete. 2024-09-13T15:43:47.565 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T15:43:47.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T15:43:47.567 INFO:tasks.ceph.osd.2:Started 2024-09-13T15:43:47.567 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T15:43:47.567 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:43:47.604 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T15:43:47.604 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 3 2024-09-13T15:43:47.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.741+0000 7f27e3f2c640 1 Processor -- start 2024-09-13T15:43:47.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.741+0000 7f27e3f2c640 1 -- start start 2024-09-13T15:43:47.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e3f2c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27dc12d330 0x7f27dc12d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:47.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e3f2c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f27dc12dd80 0x7f27dc129e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:47.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e3f2c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f27dc12a370 0x7f27dc12a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:47.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e3f2c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27dc06d5d0 con 0x7f27dc12dd80 2024-09-13T15:43:47.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e3f2c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc0971b0 con 0x7f27dc12d330 2024-09-13T15:43:47.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e3f2c640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc0979f0 con 0x7f27dc12a370 2024-09-13T15:43:47.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e24c9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f27dc12a370 0x7f27dc12a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e14c7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f27dc12dd80 0x7f27dc129e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e14c7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f27dc12dd80 0x7f27dc129e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58002/0 (socket says 172.21.15.70:58002) 2024-09-13T15:43:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e14c7640 1 -- 172.21.15.70:0/1652539467 learned_addr learned my addr 172.21.15.70:0/1652539467 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e24c9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f27dc12a370 0x7f27dc12a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:36866/0 (socket says 172.21.15.70:36866) 2024-09-13T15:43:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e1cc8640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27dc12d330 0x7f27dc12d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e14c7640 1 -- 172.21.15.70:0/1652539467 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f27dc12a370 msgr2=0x7f27dc12a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e14c7640 1 --2- 172.21.15.70:0/1652539467 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f27dc12a370 0x7f27dc12a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e14c7640 1 -- 172.21.15.70:0/1652539467 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27dc12d330 msgr2=0x7f27dc12d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e14c7640 1 --2- 172.21.15.70:0/1652539467 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27dc12d330 0x7f27dc12d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e14c7640 1 -- 172.21.15.70:0/1652539467 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc12e1a0 con 0x7f27dc12dd80 2024-09-13T15:43:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e1cc8640 1 --2- 172.21.15.70:0/1652539467 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27dc12d330 0x7f27dc12d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.742+0000 7f27e24c9640 1 --2- 172.21.15.70:0/1652539467 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f27dc12a370 0x7f27dc12a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:47.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.743+0000 7f27e14c7640 1 --2- 172.21.15.70:0/1652539467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27dc12dd80 0x7f27dc129e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009870 tx=0x7f27cc02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7f5e514a171212 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.743+0000 7f27e0cc6640 1 -- 172.21.15.70:0/1652539467 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc02adf0 con 0x7f27dc12dd80 2024-09-13T15:43:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.743+0000 7f27e0cc6640 1 -- 172.21.15.70:0/1652539467 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27cc032440 con 0x7f27dc12dd80 2024-09-13T15:43:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.743+0000 7f27e0cc6640 1 -- 172.21.15.70:0/1652539467 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc0327d0 con 0x7f27dc12dd80 2024-09-13T15:43:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.744+0000 7f27e3f2c640 1 -- 172.21.15.70:0/1652539467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27dc12dd80 msgr2=0x7f27dc129e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.744+0000 7f27e3f2c640 1 --2- 172.21.15.70:0/1652539467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27dc12dd80 0x7f27dc129e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009870 tx=0x7f27cc02a0a0 comp rx=0 tx=0).stop 2024-09-13T15:43:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.744+0000 7f27e3f2c640 1 -- 172.21.15.70:0/1652539467 shutdown_connections 2024-09-13T15:43:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.744+0000 7f27e3f2c640 1 --2- 172.21.15.70:0/1652539467 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f27dc12a370 0x7f27dc12a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.744+0000 7f27e3f2c640 1 --2- 172.21.15.70:0/1652539467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27dc12dd80 0x7f27dc129e30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.744+0000 7f27e3f2c640 1 --2- 172.21.15.70:0/1652539467 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27dc12d330 0x7f27dc12d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.744+0000 7f27e3f2c640 1 -- 172.21.15.70:0/1652539467 >> 172.21.15.70:0/1652539467 conn(0x7f27dc0a0060 msgr2=0x7f27dc0a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.744+0000 7f27e3f2c640 1 -- 172.21.15.70:0/1652539467 shutdown_connections 2024-09-13T15:43:47.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.744+0000 7f27e3f2c640 1 -- 172.21.15.70:0/1652539467 wait complete. 2024-09-13T15:43:47.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e3f2c640 1 Processor -- start 2024-09-13T15:43:47.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e3f2c640 1 -- start start 2024-09-13T15:43:47.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e3f2c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27dc12a370 0x7f27dc12edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:47.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e3f2c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f27dc12d330 0x7f27dc12f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:47.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e3f2c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27dc12dd80 0x7f27dc12f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:47.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e3f2c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc130800 con 0x7f27dc12dd80 2024-09-13T15:43:47.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e3f2c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc131950 con 0x7f27dc12a370 2024-09-13T15:43:47.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e3f2c640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc132500 con 0x7f27dc12d330 2024-09-13T15:43:47.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e1cc8640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27dc12a370 0x7f27dc12edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:47.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e14c7640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f27dc12d330 0x7f27dc12f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:47.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e24c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27dc12dd80 0x7f27dc12f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:47.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e1cc8640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27dc12a370 0x7f27dc12edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:40702/0 (socket says 172.21.15.70:40702) 2024-09-13T15:43:47.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e1cc8640 1 -- 172.21.15.70:0/1389289115 learned_addr learned my addr 172.21.15.70:0/1389289115 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:47.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e24c9640 1 -- 172.21.15.70:0/1389289115 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f27dc12d330 msgr2=0x7f27dc12f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:47.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e24c9640 1 --2- 172.21.15.70:0/1389289115 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f27dc12d330 0x7f27dc12f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:47.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e24c9640 1 -- 172.21.15.70:0/1389289115 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27dc12a370 msgr2=0x7f27dc12edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:47.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e24c9640 1 --2- 172.21.15.70:0/1389289115 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27dc12a370 0x7f27dc12edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e24c9640 1 -- 172.21.15.70:0/1389289115 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d4009a70 con 0x7f27dc12dd80 2024-09-13T15:43:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e14c7640 1 --2- 172.21.15.70:0/1389289115 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f27dc12d330 0x7f27dc12f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.745+0000 7f27e1cc8640 1 --2- 172.21.15.70:0/1389289115 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27dc12a370 0x7f27dc12edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.746+0000 7f27e24c9640 1 --2- 172.21.15.70:0/1389289115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27dc12dd80 0x7f27dc12f830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27d400efc0 tx=0x7f27d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.746+0000 7f27d27fc640 1 -- 172.21.15.70:0/1389289115 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d400cd30 con 0x7f27dc12dd80 2024-09-13T15:43:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.746+0000 7f27d27fc640 1 -- 172.21.15.70:0/1389289115 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d400f040 con 0x7f27dc12dd80 2024-09-13T15:43:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.746+0000 7f27e3f2c640 1 -- 172.21.15.70:0/1389289115 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d8009400 con 0x7f27dc12dd80 2024-09-13T15:43:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.746+0000 7f27d27fc640 1 -- 172.21.15.70:0/1389289115 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4003bd0 con 0x7f27dc12dd80 2024-09-13T15:43:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.746+0000 7f27e3f2c640 1 -- 172.21.15.70:0/1389289115 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc0094d0 con 0x7f27dc12dd80 2024-09-13T15:43:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.747+0000 7f27d27fc640 1 -- 172.21.15.70:0/1389289115 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f27d4014480 con 0x7f27dc12dd80 2024-09-13T15:43:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.747+0000 7f27e3f2c640 1 -- 172.21.15.70:0/1389289115 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27dc12d840 con 0x7f27dc12dd80 2024-09-13T15:43:47.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.747+0000 7f27d27fc640 1 -- 172.21.15.70:0/1389289115 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f27d405d330 con 0x7f27dc12dd80 2024-09-13T15:43:47.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.751+0000 7f27d27fc640 1 -- 172.21.15.70:0/1389289115 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f27d403b010 con 0x7f27dc12dd80 2024-09-13T15:43:47.895 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:47.894+0000 7f27e3f2c640 1 -- 172.21.15.70:0/1389289115 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd down", "ids": ["3"]} v 0) v1 -- 0x7f27dc1338a0 con 0x7f27dc12dd80 2024-09-13T15:43:47.896 INFO:tasks.ceph.mon.a.smithi070.stderr:2024-09-13T15:43:47.895+0000 7fee39368640 -1 mon.a@0(leader).osd e22 definitely_dead 0 2024-09-13T15:43:48.156 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-13T15:43:48.155+0000 7f3cb6b20640 -1 Falling back to public interface 2024-09-13T15:43:48.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.551+0000 7f27d27fc640 1 -- 172.21.15.70:0/1389289115 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["3"]}]=0 marked down osd.3. v23) v1 ==== 89+0+0 (secure 0 0 0) 0x7f27d4018030 con 0x7f27dc12dd80 2024-09-13T15:43:48.553 INFO:teuthology.orchestra.run.smithi070.stderr:marked down osd.3. 2024-09-13T15:43:48.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.553+0000 7f27e3f2c640 1 -- 172.21.15.70:0/1389289115 >> v1:172.21.15.70:6816/1229510400 conn(0x7f27c0047560 legacy=0x7f27c0049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:48.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.554+0000 7f27e3f2c640 1 -- 172.21.15.70:0/1389289115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27dc12dd80 msgr2=0x7f27dc12f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:48.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.554+0000 7f27e3f2c640 1 --2- 172.21.15.70:0/1389289115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27dc12dd80 0x7f27dc12f830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27d400efc0 tx=0x7f27d400c490 comp rx=0 tx=0).stop 2024-09-13T15:43:48.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.554+0000 7f27e3f2c640 1 -- 172.21.15.70:0/1389289115 shutdown_connections 2024-09-13T15:43:48.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.554+0000 7f27e3f2c640 1 --2- 172.21.15.70:0/1389289115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27dc12dd80 0x7f27dc12f830 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:48.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.554+0000 7f27e3f2c640 1 --2- 172.21.15.70:0/1389289115 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f27dc12d330 0x7f27dc12f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:48.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.554+0000 7f27e3f2c640 1 --2- 172.21.15.70:0/1389289115 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f27dc12a370 0x7f27dc12edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:48.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.554+0000 7f27e3f2c640 1 -- 172.21.15.70:0/1389289115 >> 172.21.15.70:0/1389289115 conn(0x7f27dc0a0060 msgr2=0x7f27dc09bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:48.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.554+0000 7f27e3f2c640 1 -- 172.21.15.70:0/1389289115 shutdown_connections 2024-09-13T15:43:48.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.554+0000 7f27e3f2c640 1 -- 172.21.15.70:0/1389289115 wait complete. 2024-09-13T15:43:48.566 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T15:43:48.566 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T15:43:48.568 INFO:tasks.ceph.osd.3:Started 2024-09-13T15:43:48.568 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T15:43:48.568 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T15:43:48.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.664+0000 7f0e99dbc640 1 Processor -- start 2024-09-13T15:43:48.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.664+0000 7f0e99dbc640 1 -- start start 2024-09-13T15:43:48.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.664+0000 7f0e99dbc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e9406ddf0 0x7f0e9412f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:48.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.664+0000 7f0e99dbc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e9412f960 0x7f0e94131d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:48.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.664+0000 7f0e99dbc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e9406e3a0 con 0x7f0e940953e0 2024-09-13T15:43:48.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.664+0000 7f0e99dbc640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e94095ed0 con 0x7f0e9412f960 2024-09-13T15:43:48.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.664+0000 7f0e99dbc640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e94097d40 con 0x7f0e9406ddf0 2024-09-13T15:43:48.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.665+0000 7f0e92ffd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e9406ddf0 0x7f0e9412f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:48.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.665+0000 7f0e93fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e9412f960 0x7f0e94131d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:48.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.665+0000 7f0e937fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0e940953e0 0x7f0e940957c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33704/0 (socket says 172.21.15.70:33704) 2024-09-13T15:43:48.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.665+0000 7f0e937fe640 1 -- 172.21.15.70:0/102510109 learned_addr learned my addr 172.21.15.70:0/102510109 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:48.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.665+0000 7f0e92ffd640 1 -- 172.21.15.70:0/102510109 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e9412f960 msgr2=0x7f0e94131d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:48.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.665+0000 7f0e92ffd640 1 --2- 172.21.15.70:0/102510109 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e9412f960 0x7f0e94131d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:48.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.665+0000 7f0e927fc640 1 -- 172.21.15.70:0/102510109 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215113330 0 0) 0x7f0e9406e3a0 con 0x7f0e940953e0 2024-09-13T15:43:48.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.665+0000 7f0e92ffd640 1 -- 172.21.15.70:0/102510109 >> v1:172.21.15.70:6789/0 conn(0x7f0e940953e0 legacy=0x7f0e940957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:48.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.665+0000 7f0e92ffd640 1 -- 172.21.15.70:0/102510109 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9406f110 con 0x7f0e9406ddf0 2024-09-13T15:43:48.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.665+0000 7f0e93fff640 1 --2- 172.21.15.70:0/102510109 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e9412f960 0x7f0e94131d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:48.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.666+0000 7f0e92ffd640 1 --2- 172.21.15.70:0/102510109 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e9406ddf0 0x7f0e9412f420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009800 tx=0x7f0e7c02a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e0e5e526a6edf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:48.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.666+0000 7f0e927fc640 1 -- 172.21.15.70:0/102510109 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c02ad90 con 0x7f0e9406ddf0 2024-09-13T15:43:48.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.666+0000 7f0e927fc640 1 -- 172.21.15.70:0/102510109 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7c032440 con 0x7f0e9406ddf0 2024-09-13T15:43:48.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.666+0000 7f0e927fc640 1 -- 172.21.15.70:0/102510109 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0327d0 con 0x7f0e9406ddf0 2024-09-13T15:43:48.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.666+0000 7f0e99dbc640 1 -- 172.21.15.70:0/102510109 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e9406ddf0 msgr2=0x7f0e9412f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:48.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.666+0000 7f0e99dbc640 1 --2- 172.21.15.70:0/102510109 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e9406ddf0 0x7f0e9412f420 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009800 tx=0x7f0e7c02a080 comp rx=0 tx=0).stop 2024-09-13T15:43:48.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.666+0000 7f0e99dbc640 1 -- 172.21.15.70:0/102510109 shutdown_connections 2024-09-13T15:43:48.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.666+0000 7f0e99dbc640 1 --2- 172.21.15.70:0/102510109 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e9412f960 0x7f0e94131d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:48.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.666+0000 7f0e99dbc640 1 --2- 172.21.15.70:0/102510109 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e9406ddf0 0x7f0e9412f420 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:48.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.666+0000 7f0e99dbc640 1 -- 172.21.15.70:0/102510109 >> 172.21.15.70:0/102510109 conn(0x7f0e94098b10 msgr2=0x7f0e940963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:48.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.666+0000 7f0e99dbc640 1 -- 172.21.15.70:0/102510109 shutdown_connections 2024-09-13T15:43:48.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.666+0000 7f0e99dbc640 1 -- 172.21.15.70:0/102510109 wait complete. 2024-09-13T15:43:48.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.667+0000 7f0e99dbc640 1 Processor -- start 2024-09-13T15:43:48.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.667+0000 7f0e99dbc640 1 -- start start 2024-09-13T15:43:48.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e99dbc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e9406ddf0 0x7f0e94094450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:48.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e99dbc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e940953e0 0x7f0e94094990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e99dbc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e9412f960 0x7f0e94122f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e99dbc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e94121a30 con 0x7f0e9412f960 2024-09-13T15:43:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e99dbc640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e94122a00 con 0x7f0e940953e0 2024-09-13T15:43:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e99dbc640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e94122ca0 con 0x7f0e9406ddf0 2024-09-13T15:43:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e937fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e9406ddf0 0x7f0e94094450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e92ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e940953e0 0x7f0e94094990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e93fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e9412f960 0x7f0e94122f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e937fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e9406ddf0 0x7f0e94094450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:36882/0 (socket says 172.21.15.70:36882) 2024-09-13T15:43:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e92ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e940953e0 0x7f0e94094990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:40738/0 (socket says 172.21.15.70:40738) 2024-09-13T15:43:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e937fe640 1 -- 172.21.15.70:0/3549309990 learned_addr learned my addr 172.21.15.70:0/3549309990 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e93fff640 1 -- 172.21.15.70:0/3549309990 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e9406ddf0 msgr2=0x7f0e94094450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e93fff640 1 --2- 172.21.15.70:0/3549309990 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e9406ddf0 0x7f0e94094450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:48.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e93fff640 1 -- 172.21.15.70:0/3549309990 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e940953e0 msgr2=0x7f0e94094990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:48.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e93fff640 1 --2- 172.21.15.70:0/3549309990 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e940953e0 0x7f0e94094990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:48.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e93fff640 1 -- 172.21.15.70:0/3549309990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c009a70 con 0x7f0e9412f960 2024-09-13T15:43:48.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e92ffd640 1 --2- 172.21.15.70:0/3549309990 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e940953e0 0x7f0e94094990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:48.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e937fe640 1 --2- 172.21.15.70:0/3549309990 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e9406ddf0 0x7f0e94094450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:48.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e93fff640 1 --2- 172.21.15.70:0/3549309990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e9412f960 0x7f0e94122f60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00efc0 tx=0x7f0e8c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:48.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.668+0000 7f0e8bfff640 1 -- 172.21.15.70:0/3549309990 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c00cd30 con 0x7f0e9412f960 2024-09-13T15:43:48.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.669+0000 7f0e8bfff640 1 -- 172.21.15.70:0/3549309990 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c00f040 con 0x7f0e9412f960 2024-09-13T15:43:48.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.669+0000 7f0e99dbc640 1 -- 172.21.15.70:0/3549309990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7c009400 con 0x7f0e9412f960 2024-09-13T15:43:48.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.669+0000 7f0e8bfff640 1 -- 172.21.15.70:0/3549309990 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c003bd0 con 0x7f0e9412f960 2024-09-13T15:43:48.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.669+0000 7f0e99dbc640 1 -- 172.21.15.70:0/3549309990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9406ecc0 con 0x7f0e9412f960 2024-09-13T15:43:48.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.669+0000 7f0e8bfff640 1 -- 172.21.15.70:0/3549309990 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f0e8c014480 con 0x7f0e9412f960 2024-09-13T15:43:48.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.670+0000 7f0e99dbc640 1 -- 172.21.15.70:0/3549309990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e94122150 con 0x7f0e9412f960 2024-09-13T15:43:48.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.670+0000 7f0e8bfff640 1 -- 172.21.15.70:0/3549309990 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f0e8c05d3e0 con 0x7f0e9412f960 2024-09-13T15:43:48.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.673+0000 7f0e8bfff640 1 -- 172.21.15.70:0/3549309990 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f0e94122150 con 0x7f0e9412f960 2024-09-13T15:43:48.817 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.816+0000 7f0e99dbc640 1 -- 172.21.15.70:0/3549309990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e94122150 con 0x7f0e9412f960 2024-09-13T15:43:48.817 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.816+0000 7f0e8bfff640 1 -- 172.21.15.70:0/3549309990 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+8444 (secure 0 0 0) 0x7f0e94122150 con 0x7f0e9412f960 2024-09-13T15:43:48.817 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:43:48.817 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"45bbcdad-f4fb-4639-ab42-ccda8de151dd","created":"2024-09-13T15:42:09.863982+0000","modified":"2024-09-13T15:43:48.549448+0000","last_up_change":"2024-09-13T15:43:48.549448+0000","last_in_change":"2024-09-13T15:42:11.714506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T15:42:21.000499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T15:42:26.149909+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"265c6f1d-08a7-4c43-b2a6-2417ab814825","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":32759}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":32759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":32759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":32759}]},"public_addr":"172.21.15.70:6804/32759","cluster_addr":"172.21.15.70:6805/32759","heartbeat_back_addr":"172.21.15.70:6807/32759","heartbeat_front_addr":"172.21.15.70:6806/32759","state":["exists"]},{"osd":1,"uuid":"ba4d90f9-6972-45df-800f-92ad808b06ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":20,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":32758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":32758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":32758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":32758}]},"public_addr":"172.21.15.70:6800/32758","cluster_addr":"172.21.15.70:6801/32758","heartbeat_back_addr":"172.21.15.70:6803/32758","heartbeat_front_addr":"172.21.15.70:6802/32758","state":["exists"]},{"osd":2,"uuid":"b74d8eb5-3082-4d69-a755-7f5f92dec5b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":32760}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":32760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":32760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":32760}]},"public_addr":"172.21.15.70:6812/32760","cluster_addr":"172.21.15.70:6813/32760","heartbeat_back_addr":"172.21.15.70:6815/32760","heartbeat_front_addr":"172.21.15.70:6814/32760","state":["exists"]},{"osd":3,"uuid":"02835b7d-e2cf-4ef8-87ba-b225593689d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":32769}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":32769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":32769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":32769}]},"public_addr":"172.21.15.70:6808/32769","cluster_addr":"172.21.15.70:6809/32769","heartbeat_back_addr":"172.21.15.70:6811/32769","heartbeat_front_addr":"172.21.15.70:6810/32769","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.853420+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.807491+0000","dead_epoch":21},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.925567+0000","dead_epoch":22},{"osd":3,"down_stamp":"2024-09-13T15:43:48.549448+0000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.883852+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6816/32242":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/2914106548":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3671770748":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3345804336":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3873928986":"2024-09-14T15:43:41.526957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T15:43:48.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.818+0000 7f0e99dbc640 1 -- 172.21.15.70:0/3549309990 >> v1:172.21.15.70:6816/1229510400 conn(0x7f0e74047590 legacy=0x7f0e74049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:48.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.818+0000 7f0e99dbc640 1 -- 172.21.15.70:0/3549309990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e9412f960 msgr2=0x7f0e94122f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:48.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.818+0000 7f0e99dbc640 1 --2- 172.21.15.70:0/3549309990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e9412f960 0x7f0e94122f60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00efc0 tx=0x7f0e8c00c490 comp rx=0 tx=0).stop 2024-09-13T15:43:48.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.818+0000 7f0e99dbc640 1 -- 172.21.15.70:0/3549309990 shutdown_connections 2024-09-13T15:43:48.819 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.818+0000 7f0e99dbc640 1 --2- 172.21.15.70:0/3549309990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e9412f960 0x7f0e94122f60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:48.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.818+0000 7f0e99dbc640 1 --2- 172.21.15.70:0/3549309990 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e940953e0 0x7f0e94094990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:48.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.818+0000 7f0e99dbc640 1 --2- 172.21.15.70:0/3549309990 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e9406ddf0 0x7f0e94094450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:48.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.818+0000 7f0e99dbc640 1 -- 172.21.15.70:0/3549309990 >> 172.21.15.70:0/3549309990 conn(0x7f0e94098b10 msgr2=0x7f0e9409a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:48.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.818+0000 7f0e99dbc640 1 -- 172.21.15.70:0/3549309990 shutdown_connections 2024-09-13T15:43:48.820 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:48.818+0000 7f0e99dbc640 1 -- 172.21.15.70:0/3549309990 wait complete. 2024-09-13T15:43:49.151 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-13T15:43:49.150+0000 7fcb7d83e640 -1 Falling back to public interface 2024-09-13T15:43:51.228 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-13T15:43:51.227+0000 7f9e56da4640 -1 osd.0 19 log_to_monitors true 2024-09-13T15:43:51.566 INFO:tasks.ceph.osd.0.smithi070.stderr:2024-09-13T15:43:51.565+0000 7f9e4ab41640 -1 osd.0 19 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T15:43:51.730 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-13T15:43:51.729+0000 7fe783eeb640 -1 osd.1 20 log_to_monitors true 2024-09-13T15:43:51.830 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T15:43:51.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.928+0000 7f586c1d4640 1 Processor -- start 2024-09-13T15:43:51.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.928+0000 7f586c1d4640 1 -- start start 2024-09-13T15:43:51.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f586c1d4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f586406c600 0x7f5864132710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:51.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f586c1d4640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f5864132c50 0x7f5864135040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:51.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f586c1d4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5864097940 con 0x7f586406bb10 2024-09-13T15:43:51.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f586c1d4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5864097500 con 0x7f586406c600 2024-09-13T15:43:51.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f586c1d4640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f586406d100 con 0x7f5864132c50 2024-09-13T15:43:51.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f586976f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f586406c600 0x7f5864132710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:51.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f586a771640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f5864132c50 0x7f5864135040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:51.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f586976f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f586406c600 0x7f5864132710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58026/0 (socket says 172.21.15.70:58026) 2024-09-13T15:43:51.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f586976f640 1 -- 172.21.15.70:0/859970514 learned_addr learned my addr 172.21.15.70:0/859970514 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:51.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f5869f70640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f586406bb10 0x7f586406bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33742/0 (socket says 172.21.15.70:33742) 2024-09-13T15:43:51.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f586a771640 1 -- 172.21.15.70:0/859970514 >> v2:172.21.15.70:3300/0 conn(0x7f586406c600 msgr2=0x7f5864132710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:51.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f586a771640 1 --2- 172.21.15.70:0/859970514 >> v2:172.21.15.70:3300/0 conn(0x7f586406c600 0x7f5864132710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:51.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f586a771640 1 -- 172.21.15.70:0/859970514 >> v1:172.21.15.70:6789/0 conn(0x7f586406bb10 legacy=0x7f586406bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:51.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f586a771640 1 -- 172.21.15.70:0/859970514 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586406da40 con 0x7f5864132c50 2024-09-13T15:43:51.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f5868f6e640 1 -- 172.21.15.70:0/859970514 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947675269 0 0) 0x7f5864097940 con 0x7f586406bb10 2024-09-13T15:43:51.932 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.929+0000 7f586976f640 1 --2- 172.21.15.70:0/859970514 >> v2:172.21.15.70:3300/0 conn(0x7f586406c600 0x7f5864132710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:51.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.930+0000 7f586a771640 1 --2- 172.21.15.70:0/859970514 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f5864132c50 0x7f5864135040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5854009d90 tx=0x7f585402a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79d4fc7b34c6fa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:51.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.930+0000 7f5868f6e640 1 -- 172.21.15.70:0/859970514 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5854007740 con 0x7f5864132c50 2024-09-13T15:43:51.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.930+0000 7f5868f6e640 1 -- 172.21.15.70:0/859970514 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585402d050 con 0x7f5864132c50 2024-09-13T15:43:51.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.930+0000 7f5868f6e640 1 -- 172.21.15.70:0/859970514 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5854007ac0 con 0x7f5864132c50 2024-09-13T15:43:51.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.930+0000 7f586c1d4640 1 -- 172.21.15.70:0/859970514 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f5864132c50 msgr2=0x7f5864135040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:51.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.930+0000 7f586c1d4640 1 --2- 172.21.15.70:0/859970514 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f5864132c50 0x7f5864135040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5854009d90 tx=0x7f585402a530 comp rx=0 tx=0).stop 2024-09-13T15:43:51.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.931+0000 7f586c1d4640 1 -- 172.21.15.70:0/859970514 shutdown_connections 2024-09-13T15:43:51.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.931+0000 7f586c1d4640 1 --2- 172.21.15.70:0/859970514 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f5864132c50 0x7f5864135040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:51.936 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.931+0000 7f586c1d4640 1 --2- 172.21.15.70:0/859970514 >> v2:172.21.15.70:3300/0 conn(0x7f586406c600 0x7f5864132710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:51.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.931+0000 7f586c1d4640 1 -- 172.21.15.70:0/859970514 >> 172.21.15.70:0/859970514 conn(0x7f586409fe20 msgr2=0x7f58640a0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:51.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.931+0000 7f586c1d4640 1 -- 172.21.15.70:0/859970514 shutdown_connections 2024-09-13T15:43:51.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.931+0000 7f586c1d4640 1 -- 172.21.15.70:0/859970514 wait complete. 2024-09-13T15:43:51.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.931+0000 7f586c1d4640 1 Processor -- start 2024-09-13T15:43:51.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.931+0000 7f586c1d4640 1 -- start start 2024-09-13T15:43:51.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f586c1d4640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f586406bb10 0x7f586412b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:51.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f586c1d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f586406c600 0x7f586412bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:51.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f586c1d4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5864132c50 0x7f586412c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:51.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f586c1d4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586412dc30 con 0x7f586406c600 2024-09-13T15:43:51.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f586c1d4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586412f060 con 0x7f5864132c50 2024-09-13T15:43:51.937 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f586c1d4640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f586412d210 con 0x7f586406bb10 2024-09-13T15:43:51.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f5869f70640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f586406bb10 0x7f586412b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:51.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f586976f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f586406c600 0x7f586412bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:51.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f586a771640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5864132c50 0x7f586412c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:51.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f5869f70640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f586406bb10 0x7f586412b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:36910/0 (socket says 172.21.15.70:36910) 2024-09-13T15:43:51.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f5869f70640 1 -- 172.21.15.70:0/3764850464 learned_addr learned my addr 172.21.15.70:0/3764850464 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:51.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f5869f70640 1 -- 172.21.15.70:0/3764850464 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5864132c50 msgr2=0x7f586412c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:51.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f5869f70640 1 --2- 172.21.15.70:0/3764850464 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5864132c50 0x7f586412c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:51.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f5869f70640 1 -- 172.21.15.70:0/3764850464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f586406c600 msgr2=0x7f586412bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:51.938 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f5869f70640 1 --2- 172.21.15.70:0/3764850464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f586406c600 0x7f586412bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:51.939 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f5869f70640 1 -- 172.21.15.70:0/3764850464 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58580094d0 con 0x7f586406bb10 2024-09-13T15:43:51.939 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f586976f640 1 --2- 172.21.15.70:0/3764850464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f586406c600 0x7f586412bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:51.939 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.932+0000 7f5869f70640 1 --2- 172.21.15.70:0/3764850464 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f586406bb10 0x7f586412b750 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f585c00c4a0 tx=0x7f585c00c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:51.939 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.933+0000 7f58627fc640 1 -- 172.21.15.70:0/3764850464 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c0074e0 con 0x7f586406bb10 2024-09-13T15:43:51.939 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.933+0000 7f58627fc640 1 -- 172.21.15.70:0/3764850464 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585c007640 con 0x7f586406bb10 2024-09-13T15:43:51.939 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.933+0000 7f58627fc640 1 -- 172.21.15.70:0/3764850464 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c004a00 con 0x7f586406bb10 2024-09-13T15:43:51.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.933+0000 7f586c1d4640 1 -- 172.21.15.70:0/3764850464 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5854009a70 con 0x7f586406bb10 2024-09-13T15:43:51.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.933+0000 7f586c1d4640 1 -- 172.21.15.70:0/3764850464 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5864132230 con 0x7f586406bb10 2024-09-13T15:43:51.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.933+0000 7f586c1d4640 1 -- 172.21.15.70:0/3764850464 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586406bef0 con 0x7f586406bb10 2024-09-13T15:43:51.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.934+0000 7f58627fc640 1 -- 172.21.15.70:0/3764850464 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f585c004b60 con 0x7f586406bb10 2024-09-13T15:43:51.940 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.934+0000 7f58627fc640 1 -- 172.21.15.70:0/3764850464 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f585c05ba40 con 0x7f586406bb10 2024-09-13T15:43:51.941 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:51.937+0000 7f58627fc640 1 -- 172.21.15.70:0/3764850464 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f585c05de60 con 0x7f586406bb10 2024-09-13T15:43:52.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:52.084+0000 7f586c1d4640 1 -- 172.21.15.70:0/3764850464 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f586412a660 con 0x7f586406bb10 2024-09-13T15:43:52.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:52.085+0000 7f58627fc640 1 -- 172.21.15.70:0/3764850464 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+8444 (secure 0 0 0) 0x7f585c018020 con 0x7f586406bb10 2024-09-13T15:43:52.086 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:43:52.086 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":24,"fsid":"45bbcdad-f4fb-4639-ab42-ccda8de151dd","created":"2024-09-13T15:42:09.863982+0000","modified":"2024-09-13T15:43:51.557312+0000","last_up_change":"2024-09-13T15:43:48.549448+0000","last_in_change":"2024-09-13T15:42:11.714506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T15:42:21.000499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T15:42:26.149909+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"265c6f1d-08a7-4c43-b2a6-2417ab814825","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":32759}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":32759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":32759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":32759}]},"public_addr":"172.21.15.70:6804/32759","cluster_addr":"172.21.15.70:6805/32759","heartbeat_back_addr":"172.21.15.70:6807/32759","heartbeat_front_addr":"172.21.15.70:6806/32759","state":["exists"]},{"osd":1,"uuid":"ba4d90f9-6972-45df-800f-92ad808b06ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":20,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":32758}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":32758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":32758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":32758}]},"public_addr":"172.21.15.70:6800/32758","cluster_addr":"172.21.15.70:6801/32758","heartbeat_back_addr":"172.21.15.70:6803/32758","heartbeat_front_addr":"172.21.15.70:6802/32758","state":["exists"]},{"osd":2,"uuid":"b74d8eb5-3082-4d69-a755-7f5f92dec5b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":32760}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":32760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":32760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":32760}]},"public_addr":"172.21.15.70:6812/32760","cluster_addr":"172.21.15.70:6813/32760","heartbeat_back_addr":"172.21.15.70:6815/32760","heartbeat_front_addr":"172.21.15.70:6814/32760","state":["exists"]},{"osd":3,"uuid":"02835b7d-e2cf-4ef8-87ba-b225593689d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":32769}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":32769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":32769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":32769}]},"public_addr":"172.21.15.70:6808/32769","cluster_addr":"172.21.15.70:6809/32769","heartbeat_back_addr":"172.21.15.70:6811/32769","heartbeat_front_addr":"172.21.15.70:6810/32769","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.853420+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.807491+0000","dead_epoch":21},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.925567+0000","dead_epoch":22},{"osd":3,"down_stamp":"2024-09-13T15:43:48.549448+0000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.883852+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6816/32242":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/2914106548":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3671770748":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3345804336":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3873928986":"2024-09-14T15:43:41.526957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T15:43:52.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:52.086+0000 7f586c1d4640 1 -- 172.21.15.70:0/3764850464 >> v1:172.21.15.70:6816/1229510400 conn(0x7f5850047560 legacy=0x7f5850049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:52.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:52.086+0000 7f586c1d4640 1 -- 172.21.15.70:0/3764850464 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f586406bb10 msgr2=0x7f586412b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:52.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:52.086+0000 7f586c1d4640 1 --2- 172.21.15.70:0/3764850464 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f586406bb10 0x7f586412b750 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f585c00c4a0 tx=0x7f585c00c960 comp rx=0 tx=0).stop 2024-09-13T15:43:52.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:52.086+0000 7f586c1d4640 1 -- 172.21.15.70:0/3764850464 shutdown_connections 2024-09-13T15:43:52.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:52.086+0000 7f586c1d4640 1 --2- 172.21.15.70:0/3764850464 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5864132c50 0x7f586412c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:52.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:52.086+0000 7f586c1d4640 1 --2- 172.21.15.70:0/3764850464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f586406c600 0x7f586412bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:52.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:52.086+0000 7f586c1d4640 1 --2- 172.21.15.70:0/3764850464 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f586406bb10 0x7f586412b750 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:52.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:52.086+0000 7f586c1d4640 1 -- 172.21.15.70:0/3764850464 >> 172.21.15.70:0/3764850464 conn(0x7f586409fe20 msgr2=0x7f586409bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:52.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:52.086+0000 7f586c1d4640 1 -- 172.21.15.70:0/3764850464 shutdown_connections 2024-09-13T15:43:52.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:52.086+0000 7f586c1d4640 1 -- 172.21.15.70:0/3764850464 wait complete. 2024-09-13T15:43:52.570 INFO:tasks.ceph.osd.1.smithi070.stderr:2024-09-13T15:43:52.569+0000 7fe777c88640 -1 osd.1 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T15:43:52.687 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-13T15:43:52.686+0000 7f3cb6b20640 -1 osd.2 21 log_to_monitors true 2024-09-13T15:43:53.574 INFO:tasks.ceph.osd.2.smithi070.stderr:2024-09-13T15:43:53.573+0000 7f3caa8bd640 -1 osd.2 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T15:43:53.677 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-13T15:43:53.676+0000 7fcb7d83e640 -1 osd.3 22 log_to_monitors true 2024-09-13T15:43:54.581 INFO:tasks.ceph.osd.3.smithi070.stderr:2024-09-13T15:43:54.580+0000 7fcb715db640 -1 osd.3 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T15:43:55.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T15:43:55.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.207+0000 7fbfa197d640 1 Processor -- start 2024-09-13T15:43:55.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.208+0000 7fbfa197d640 1 -- start start 2024-09-13T15:43:55.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.208+0000 7fbfa197d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf9c12dee0 0x7fbf9c12e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:55.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.208+0000 7fbfa197d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf9c12e800 0x7fbf9c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:55.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.208+0000 7fbfa197d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9c097fc0 con 0x7fbf9c06dc50 2024-09-13T15:43:55.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.208+0000 7fbfa197d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c09ead0 con 0x7fbf9c12e800 2024-09-13T15:43:55.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.208+0000 7fbfa197d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c067970 con 0x7fbf9c12dee0 2024-09-13T15:43:55.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.208+0000 7fbf9b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf9c12e800 0x7fbf9c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:55.210 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.208+0000 7fbf9affd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf9c12dee0 0x7fbf9c12e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:55.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.208+0000 7fbf9b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf9c12e800 0x7fbf9c06c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40720/0 (socket says 172.21.15.70:40720) 2024-09-13T15:43:55.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.208+0000 7fbf9affd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf9c12dee0 0x7fbf9c12e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59254/0 (socket says 172.21.15.70:59254) 2024-09-13T15:43:55.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.208+0000 7fbf9b7fe640 1 -- 172.21.15.70:0/2528157096 learned_addr learned my addr 172.21.15.70:0/2528157096 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:55.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.208+0000 7fbf9a7fc640 1 --1- 172.21.15.70:0/2528157096 >> v1:172.21.15.70:6789/0 conn(0x7fbf9c06dc50 0x7fbf9c06be90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60406/0 (socket says 172.21.15.70:60406) 2024-09-13T15:43:55.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.208+0000 7fbf9b7fe640 1 -- 172.21.15.70:0/2528157096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf9c12dee0 msgr2=0x7fbf9c12e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:55.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.208+0000 7fbf9b7fe640 1 --2- 172.21.15.70:0/2528157096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf9c12dee0 0x7fbf9c12e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:55.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.208+0000 7fbf9b7fe640 1 -- 172.21.15.70:0/2528157096 >> v1:172.21.15.70:6789/0 conn(0x7fbf9c06dc50 legacy=0x7fbf9c06be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:55.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.209+0000 7fbf9b7fe640 1 -- 172.21.15.70:0/2528157096 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c06e870 con 0x7fbf9c12e800 2024-09-13T15:43:55.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.209+0000 7fbf99ffb640 1 -- 172.21.15.70:0/2528157096 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15013233 0 0) 0x7fbf9c097fc0 con 0x7fbf9c06dc50 2024-09-13T15:43:55.211 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.209+0000 7fbf9affd640 1 --2- 172.21.15.70:0/2528157096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf9c12dee0 0x7fbf9c12e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:55.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.209+0000 7fbf9b7fe640 1 --2- 172.21.15.70:0/2528157096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c12e800 0x7fbf9c06c5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009e50 tx=0x7fbf9402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d627255bef82330 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:55.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.209+0000 7fbf99ffb640 1 -- 172.21.15.70:0/2528157096 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94031600 con 0x7fbf9c12e800 2024-09-13T15:43:55.214 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.209+0000 7fbf99ffb640 1 -- 172.21.15.70:0/2528157096 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9402d050 con 0x7fbf9c12e800 2024-09-13T15:43:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.209+0000 7fbf99ffb640 1 -- 172.21.15.70:0/2528157096 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94031980 con 0x7fbf9c12e800 2024-09-13T15:43:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.210+0000 7fbfa197d640 1 -- 172.21.15.70:0/2528157096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c12e800 msgr2=0x7fbf9c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.210+0000 7fbfa197d640 1 --2- 172.21.15.70:0/2528157096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c12e800 0x7fbf9c06c5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009e50 tx=0x7fbf9402a5f0 comp rx=0 tx=0).stop 2024-09-13T15:43:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.210+0000 7fbfa197d640 1 -- 172.21.15.70:0/2528157096 shutdown_connections 2024-09-13T15:43:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.210+0000 7fbfa197d640 1 --2- 172.21.15.70:0/2528157096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c12e800 0x7fbf9c06c5a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.210+0000 7fbfa197d640 1 --2- 172.21.15.70:0/2528157096 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf9c12dee0 0x7fbf9c12e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.210+0000 7fbfa197d640 1 -- 172.21.15.70:0/2528157096 >> 172.21.15.70:0/2528157096 conn(0x7fbf9c09dfb0 msgr2=0x7fbf9c09e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.210+0000 7fbfa197d640 1 -- 172.21.15.70:0/2528157096 shutdown_connections 2024-09-13T15:43:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.210+0000 7fbfa197d640 1 -- 172.21.15.70:0/2528157096 wait complete. 2024-09-13T15:43:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.211+0000 7fbfa197d640 1 Processor -- start 2024-09-13T15:43:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.211+0000 7fbfa197d640 1 -- start start 2024-09-13T15:43:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.211+0000 7fbfa197d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbf9c06dc50 0x7fbf9c1c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.211+0000 7fbfa197d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf9c12dee0 0x7fbf9c1c2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbfa197d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c12e800 0x7fbf9c1c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbfa197d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c12db20 con 0x7fbf9c12e800 2024-09-13T15:43:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbfa197d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c097430 con 0x7fbf9c12dee0 2024-09-13T15:43:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbfa197d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c094870 con 0x7fbf9c06dc50 2024-09-13T15:43:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf9affd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbf9c06dc50 0x7fbf9c1c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf9b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c12e800 0x7fbf9c1c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf9a7fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf9c12dee0 0x7fbf9c1c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf9b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c12e800 0x7fbf9c1c2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40730/0 (socket says 172.21.15.70:40730) 2024-09-13T15:43:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf9affd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbf9c06dc50 0x7fbf9c1c21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58490/0 (socket says 172.21.15.70:58490) 2024-09-13T15:43:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf9b7fe640 1 -- 172.21.15.70:0/1607734511 learned_addr learned my addr 172.21.15.70:0/1607734511 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf9affd640 1 -- 172.21.15.70:0/1607734511 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf9c12dee0 msgr2=0x7fbf9c1c2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf9affd640 1 --2- 172.21.15.70:0/1607734511 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf9c12dee0 0x7fbf9c1c2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:55.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf9affd640 1 -- 172.21.15.70:0/1607734511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c12e800 msgr2=0x7fbf9c1c2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:55.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf9affd640 1 --2- 172.21.15.70:0/1607734511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c12e800 0x7fbf9c1c2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:55.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf9affd640 1 -- 172.21.15.70:0/1607734511 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c009400 con 0x7fbf9c06dc50 2024-09-13T15:43:55.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf9a7fc640 1 --2- 172.21.15.70:0/1607734511 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf9c12dee0 0x7fbf9c1c2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:55.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf9b7fe640 1 --2- 172.21.15.70:0/1607734511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c12e800 0x7fbf9c1c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:55.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf9affd640 1 --2- 172.21.15.70:0/1607734511 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbf9c06dc50 0x7fbf9c1c21c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00e900 tx=0x7fbf8c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:55.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf93fff640 1 -- 172.21.15.70:0/1607734511 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c00ca00 con 0x7fbf9c06dc50 2024-09-13T15:43:55.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf93fff640 1 -- 172.21.15.70:0/1607734511 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8c00cb60 con 0x7fbf9c06dc50 2024-09-13T15:43:55.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbfa197d640 1 -- 172.21.15.70:0/1607734511 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf94009b30 con 0x7fbf9c06dc50 2024-09-13T15:43:55.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbf93fff640 1 -- 172.21.15.70:0/1607734511 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c013470 con 0x7fbf9c06dc50 2024-09-13T15:43:55.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.212+0000 7fbfa197d640 1 -- 172.21.15.70:0/1607734511 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c096310 con 0x7fbf9c06dc50 2024-09-13T15:43:55.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.213+0000 7fbf93fff640 1 -- 172.21.15.70:0/1607734511 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fbf8c0135d0 con 0x7fbf9c06dc50 2024-09-13T15:43:55.217 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.213+0000 7fbfa197d640 1 -- 172.21.15.70:0/1607734511 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c1b4790 con 0x7fbf9c06dc50 2024-09-13T15:43:55.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.214+0000 7fbf93fff640 1 -- 172.21.15.70:0/1607734511 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fbf8c05caf0 con 0x7fbf9c06dc50 2024-09-13T15:43:55.218 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.217+0000 7fbf93fff640 1 -- 172.21.15.70:0/1607734511 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fbf8c03a710 con 0x7fbf9c06dc50 2024-09-13T15:43:55.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.352+0000 7fbfa197d640 1 -- 172.21.15.70:0/1607734511 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf9c0968c0 con 0x7fbf9c06dc50 2024-09-13T15:43:55.353 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.352+0000 7fbf93fff640 1 -- 172.21.15.70:0/1607734511 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+9003 (secure 0 0 0) 0x7fbf8c017030 con 0x7fbf9c06dc50 2024-09-13T15:43:55.354 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:43:55.354 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"fsid":"45bbcdad-f4fb-4639-ab42-ccda8de151dd","created":"2024-09-13T15:42:09.863982+0000","modified":"2024-09-13T15:43:54.567645+0000","last_up_change":"2024-09-13T15:43:54.567645+0000","last_in_change":"2024-09-13T15:42:11.714506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T15:42:21.000499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T15:42:26.149909+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":0.75,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"265c6f1d-08a7-4c43-b2a6-2417ab814825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":25,"up_thru":25,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":2246103037}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":2246103037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":2246103037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":2246103037}]},"public_addr":"172.21.15.70:6800/2246103037","cluster_addr":"172.21.15.70:6801/2246103037","heartbeat_back_addr":"172.21.15.70:6803/2246103037","heartbeat_front_addr":"172.21.15.70:6802/2246103037","state":["exists","up"]},{"osd":1,"uuid":"ba4d90f9-6972-45df-800f-92ad808b06ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":26,"up_thru":26,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":853418053}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":853418053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":853418053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":853418053}]},"public_addr":"172.21.15.70:6804/853418053","cluster_addr":"172.21.15.70:6805/853418053","heartbeat_back_addr":"172.21.15.70:6807/853418053","heartbeat_front_addr":"172.21.15.70:6806/853418053","state":["exists","up"]},{"osd":2,"uuid":"b74d8eb5-3082-4d69-a755-7f5f92dec5b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":27,"up_thru":13,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":2115999109}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":2115999109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":2115999109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":2115999109}]},"public_addr":"172.21.15.70:6808/2115999109","cluster_addr":"172.21.15.70:6809/2115999109","heartbeat_back_addr":"172.21.15.70:6811/2115999109","heartbeat_front_addr":"172.21.15.70:6810/2115999109","state":["exists","up"]},{"osd":3,"uuid":"02835b7d-e2cf-4ef8-87ba-b225593689d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":32769}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":32769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":32769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":32769}]},"public_addr":"172.21.15.70:6808/32769","cluster_addr":"172.21.15.70:6809/32769","heartbeat_back_addr":"172.21.15.70:6811/32769","heartbeat_front_addr":"172.21.15.70:6810/32769","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.853420+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.807491+0000","dead_epoch":21},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.925567+0000","dead_epoch":22},{"osd":3,"down_stamp":"2024-09-13T15:43:48.549448+0000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.883852+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6816/32242":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/2914106548":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3671770748":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3345804336":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3873928986":"2024-09-14T15:43:41.526957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T15:43:55.355 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.354+0000 7fbfa197d640 1 -- 172.21.15.70:0/1607734511 >> v1:172.21.15.70:6816/1229510400 conn(0x7fbf7c047560 legacy=0x7fbf7c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:55.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.354+0000 7fbfa197d640 1 -- 172.21.15.70:0/1607734511 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbf9c06dc50 msgr2=0x7fbf9c1c21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:55.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.354+0000 7fbfa197d640 1 --2- 172.21.15.70:0/1607734511 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbf9c06dc50 0x7fbf9c1c21c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00e900 tx=0x7fbf8c00edc0 comp rx=0 tx=0).stop 2024-09-13T15:43:55.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.354+0000 7fbfa197d640 1 -- 172.21.15.70:0/1607734511 shutdown_connections 2024-09-13T15:43:55.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.354+0000 7fbfa197d640 1 --2- 172.21.15.70:0/1607734511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf9c12e800 0x7fbf9c1c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:55.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.354+0000 7fbfa197d640 1 --2- 172.21.15.70:0/1607734511 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf9c12dee0 0x7fbf9c1c2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:55.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.354+0000 7fbfa197d640 1 --2- 172.21.15.70:0/1607734511 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbf9c06dc50 0x7fbf9c1c21c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:55.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.354+0000 7fbfa197d640 1 -- 172.21.15.70:0/1607734511 >> 172.21.15.70:0/1607734511 conn(0x7fbf9c09dfb0 msgr2=0x7fbf9c1b4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:55.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.354+0000 7fbfa197d640 1 -- 172.21.15.70:0/1607734511 shutdown_connections 2024-09-13T15:43:55.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:55.354+0000 7fbfa197d640 1 -- 172.21.15.70:0/1607734511 wait complete. 2024-09-13T15:43:58.368 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T15:43:58.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.476+0000 7f4fcd468640 1 Processor -- start 2024-09-13T15:43:58.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.476+0000 7f4fcd468640 1 -- start start 2024-09-13T15:43:58.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.476+0000 7f4fcd468640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc806c070 0x7f4fc806c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:58.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.476+0000 7f4fcd468640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4fc806e050 0x7f4fc806c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:58.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.476+0000 7f4fcd468640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4fc806ced0 0x7f4fc8133d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:58.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.476+0000 7f4fcd468640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc806d7f0 con 0x7f4fc806ced0 2024-09-13T15:43:58.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.476+0000 7f4fcd468640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc806da30 con 0x7f4fc806c070 2024-09-13T15:43:58.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.476+0000 7f4fcd468640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc806d3a0 con 0x7f4fc806e050 2024-09-13T15:43:58.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.476+0000 7f4fc6ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc806c070 0x7f4fc806c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.476+0000 7f4fc67fc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4fc806e050 0x7f4fc806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.476+0000 7f4fc77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4fc806ced0 0x7f4fc8133d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.476+0000 7f4fc67fc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4fc806e050 0x7f4fc806c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58492/0 (socket says 172.21.15.70:58492) 2024-09-13T15:43:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.476+0000 7f4fc77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4fc806ced0 0x7f4fc8133d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40740/0 (socket says 172.21.15.70:40740) 2024-09-13T15:43:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.476+0000 7f4fc6ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc806c070 0x7f4fc806c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59278/0 (socket says 172.21.15.70:59278) 2024-09-13T15:43:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.476+0000 7f4fc67fc640 1 -- 172.21.15.70:0/2874803668 learned_addr learned my addr 172.21.15.70:0/2874803668 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.477+0000 7f4fc6ffd640 1 -- 172.21.15.70:0/2874803668 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4fc806e050 msgr2=0x7f4fc806c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.477+0000 7f4fc6ffd640 1 --2- 172.21.15.70:0/2874803668 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4fc806e050 0x7f4fc806c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.477+0000 7f4fc6ffd640 1 -- 172.21.15.70:0/2874803668 >> v2:172.21.15.70:3300/0 conn(0x7f4fc806ced0 msgr2=0x7f4fc8133d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.477+0000 7f4fc6ffd640 1 --2- 172.21.15.70:0/2874803668 >> v2:172.21.15.70:3300/0 conn(0x7f4fc806ced0 0x7f4fc8133d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.477+0000 7f4fc6ffd640 1 -- 172.21.15.70:0/2874803668 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc8134270 con 0x7f4fc806c070 2024-09-13T15:43:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.477+0000 7f4fc77fe640 1 --2- 172.21.15.70:0/2874803668 >> v2:172.21.15.70:3300/0 conn(0x7f4fc806ced0 0x7f4fc8133d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:58.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.477+0000 7f4fc6ffd640 1 --2- 172.21.15.70:0/2874803668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc806c070 0x7f4fc806c450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80097d0 tx=0x7f4fb802a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eed87db9d3051fa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.478+0000 7f4fc5ffb640 1 -- 172.21.15.70:0/2874803668 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb802ad10 con 0x7f4fc806c070 2024-09-13T15:43:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.478+0000 7f4fc5ffb640 1 -- 172.21.15.70:0/2874803668 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb802ae70 con 0x7f4fc806c070 2024-09-13T15:43:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.478+0000 7f4fc5ffb640 1 -- 172.21.15.70:0/2874803668 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8032640 con 0x7f4fc806c070 2024-09-13T15:43:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.478+0000 7f4fcd468640 1 -- 172.21.15.70:0/2874803668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc806c070 msgr2=0x7f4fc806c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.478+0000 7f4fcd468640 1 --2- 172.21.15.70:0/2874803668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc806c070 0x7f4fc806c450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80097d0 tx=0x7f4fb802a000 comp rx=0 tx=0).stop 2024-09-13T15:43:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.478+0000 7f4fcd468640 1 -- 172.21.15.70:0/2874803668 shutdown_connections 2024-09-13T15:43:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.478+0000 7f4fcd468640 1 --2- 172.21.15.70:0/2874803668 >> v2:172.21.15.70:3300/0 conn(0x7f4fc806ced0 0x7f4fc8133d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.478+0000 7f4fcd468640 1 --2- 172.21.15.70:0/2874803668 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4fc806e050 0x7f4fc806c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.478+0000 7f4fcd468640 1 --2- 172.21.15.70:0/2874803668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc806c070 0x7f4fc806c450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.478+0000 7f4fcd468640 1 -- 172.21.15.70:0/2874803668 >> 172.21.15.70:0/2874803668 conn(0x7f4fc809dd70 msgr2=0x7f4fc809e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.478+0000 7f4fcd468640 1 -- 172.21.15.70:0/2874803668 shutdown_connections 2024-09-13T15:43:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.478+0000 7f4fcd468640 1 -- 172.21.15.70:0/2874803668 wait complete. 2024-09-13T15:43:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.479+0000 7f4fcd468640 1 Processor -- start 2024-09-13T15:43:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.479+0000 7f4fcd468640 1 -- start start 2024-09-13T15:43:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fcd468640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4fc806c070 0x7f4fc8122ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fcd468640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc806ced0 0x7f4fc8123400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fcd468640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc806e050 0x7f4fc8123940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fcd468640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc8124d00 con 0x7f4fc806e050 2024-09-13T15:43:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fcd468640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc81261d0 con 0x7f4fc806ced0 2024-09-13T15:43:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fcd468640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc81244f0 con 0x7f4fc806c070 2024-09-13T15:43:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fc6ffd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4fc806c070 0x7f4fc8122ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fc67fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc806ced0 0x7f4fc8123400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fc77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc806e050 0x7f4fc8123940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:58.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fc6ffd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4fc806c070 0x7f4fc8122ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58500/0 (socket says 172.21.15.70:58500) 2024-09-13T15:43:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fc6ffd640 1 -- 172.21.15.70:0/586201618 learned_addr learned my addr 172.21.15.70:0/586201618 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fc67fc640 1 -- 172.21.15.70:0/586201618 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4fc806c070 msgr2=0x7f4fc8122ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fc67fc640 1 --2- 172.21.15.70:0/586201618 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4fc806c070 0x7f4fc8122ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fc67fc640 1 -- 172.21.15.70:0/586201618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc806e050 msgr2=0x7f4fc8123940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fc67fc640 1 --2- 172.21.15.70:0/586201618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc806e050 0x7f4fc8123940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fc67fc640 1 -- 172.21.15.70:0/586201618 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc0009a70 con 0x7f4fc806ced0 2024-09-13T15:43:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fc6ffd640 1 --2- 172.21.15.70:0/586201618 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4fc806c070 0x7f4fc8122ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fc77fe640 1 --2- 172.21.15.70:0/586201618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc806e050 0x7f4fc8123940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fc67fc640 1 --2- 172.21.15.70:0/586201618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc806ced0 0x7f4fc8123400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00e820 tx=0x7f4fbc00ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4faf7fe640 1 -- 172.21.15.70:0/586201618 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc00ee70 con 0x7f4fc806ced0 2024-09-13T15:43:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4faf7fe640 1 -- 172.21.15.70:0/586201618 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fbc00c8e0 con 0x7f4fc806ced0 2024-09-13T15:43:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4fcd468640 1 -- 172.21.15.70:0/586201618 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb8009400 con 0x7f4fc806ced0 2024-09-13T15:43:58.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.480+0000 7f4faf7fe640 1 -- 172.21.15.70:0/586201618 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc00cc50 con 0x7f4fc806ced0 2024-09-13T15:43:58.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.481+0000 7f4fcd468640 1 -- 172.21.15.70:0/586201618 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc81257b0 con 0x7f4fc806ced0 2024-09-13T15:43:58.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.481+0000 7f4faf7fe640 1 -- 172.21.15.70:0/586201618 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f4fbc013470 con 0x7f4fc806ced0 2024-09-13T15:43:58.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.482+0000 7f4fcd468640 1 -- 172.21.15.70:0/586201618 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc806c450 con 0x7f4fc806ced0 2024-09-13T15:43:58.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.482+0000 7f4faf7fe640 1 -- 172.21.15.70:0/586201618 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f4fbc05c8e0 con 0x7f4fc806ced0 2024-09-13T15:43:58.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.486+0000 7f4faf7fe640 1 -- 172.21.15.70:0/586201618 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f4fbc03a480 con 0x7f4fc806ced0 2024-09-13T15:43:58.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.623+0000 7f4fcd468640 1 -- 172.21.15.70:0/586201618 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fc81251b0 con 0x7f4fc806ced0 2024-09-13T15:43:58.625 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.623+0000 7f4faf7fe640 1 -- 172.21.15.70:0/586201618 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+9051 (secure 0 0 0) 0x7f4fbc03daa0 con 0x7f4fc806ced0 2024-09-13T15:43:58.626 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:43:58.626 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":29,"fsid":"45bbcdad-f4fb-4639-ab42-ccda8de151dd","created":"2024-09-13T15:42:09.863982+0000","modified":"2024-09-13T15:43:56.576725+0000","last_up_change":"2024-09-13T15:43:55.573914+0000","last_in_change":"2024-09-13T15:42:11.714506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"quincy","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T15:42:21.000499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T15:42:26.149909+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"265c6f1d-08a7-4c43-b2a6-2417ab814825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":25,"up_thru":25,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":2246103037}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":2246103037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":2246103037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":2246103037}]},"public_addr":"172.21.15.70:6800/2246103037","cluster_addr":"172.21.15.70:6801/2246103037","heartbeat_back_addr":"172.21.15.70:6803/2246103037","heartbeat_front_addr":"172.21.15.70:6802/2246103037","state":["exists","up"]},{"osd":1,"uuid":"ba4d90f9-6972-45df-800f-92ad808b06ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":26,"up_thru":26,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":853418053}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":853418053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":853418053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":853418053}]},"public_addr":"172.21.15.70:6804/853418053","cluster_addr":"172.21.15.70:6805/853418053","heartbeat_back_addr":"172.21.15.70:6807/853418053","heartbeat_front_addr":"172.21.15.70:6806/853418053","state":["exists","up"]},{"osd":2,"uuid":"b74d8eb5-3082-4d69-a755-7f5f92dec5b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":27,"up_thru":28,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":2115999109}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":2115999109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":2115999109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":2115999109}]},"public_addr":"172.21.15.70:6808/2115999109","cluster_addr":"172.21.15.70:6809/2115999109","heartbeat_back_addr":"172.21.15.70:6811/2115999109","heartbeat_front_addr":"172.21.15.70:6810/2115999109","state":["exists","up"]},{"osd":3,"uuid":"02835b7d-e2cf-4ef8-87ba-b225593689d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":28,"up_thru":28,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":2095410362}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":2095410362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":2095410362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":2095410362}]},"public_addr":"172.21.15.70:6812/2095410362","cluster_addr":"172.21.15.70:6813/2095410362","heartbeat_back_addr":"172.21.15.70:6815/2095410362","heartbeat_front_addr":"172.21.15.70:6814/2095410362","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.853420+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.807491+0000","dead_epoch":21},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.925567+0000","dead_epoch":22},{"osd":3,"down_stamp":"2024-09-13T15:43:48.549448+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.883852+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6816/32242":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/2914106548":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3671770748":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3345804336":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3873928986":"2024-09-14T15:43:41.526957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T15:43:58.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.625+0000 7f4fcd468640 1 -- 172.21.15.70:0/586201618 >> v1:172.21.15.70:6816/1229510400 conn(0x7f4fb4047560 legacy=0x7f4fb4049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:58.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.625+0000 7f4fcd468640 1 -- 172.21.15.70:0/586201618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc806ced0 msgr2=0x7f4fc8123400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:58.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.625+0000 7f4fcd468640 1 --2- 172.21.15.70:0/586201618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc806ced0 0x7f4fc8123400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00e820 tx=0x7f4fbc00ec20 comp rx=0 tx=0).stop 2024-09-13T15:43:58.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.625+0000 7f4fcd468640 1 -- 172.21.15.70:0/586201618 shutdown_connections 2024-09-13T15:43:58.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.625+0000 7f4fcd468640 1 --2- 172.21.15.70:0/586201618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc806e050 0x7f4fc8123940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.625+0000 7f4fcd468640 1 --2- 172.21.15.70:0/586201618 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f4fc806ced0 0x7f4fc8123400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.625+0000 7f4fcd468640 1 --2- 172.21.15.70:0/586201618 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f4fc806c070 0x7f4fc8122ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.625+0000 7f4fcd468640 1 -- 172.21.15.70:0/586201618 >> 172.21.15.70:0/586201618 conn(0x7f4fc809dd70 msgr2=0x7f4fc8098700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:58.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.625+0000 7f4fcd468640 1 -- 172.21.15.70:0/586201618 shutdown_connections 2024-09-13T15:43:58.628 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.625+0000 7f4fcd468640 1 -- 172.21.15.70:0/586201618 wait complete. 2024-09-13T15:43:58.639 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T15:43:58.639 INFO:teuthology.run_tasks:Running task exec... 2024-09-13T15:43:58.648 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T15:43:58.648 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T15:43:58.648 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph versions' 2024-09-13T15:43:58.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.739+0000 7fe6fa45a640 1 Processor -- start 2024-09-13T15:43:58.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.739+0000 7fe6fa45a640 1 -- start start 2024-09-13T15:43:58.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.739+0000 7fe6fa45a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6f406b860 0x7fe6f406bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:58.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.739+0000 7fe6fa45a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe6f406c180 0x7fe6f412d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:58.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.739+0000 7fe6fa45a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f4066060 con 0x7fe6f406c560 2024-09-13T15:43:58.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.739+0000 7fe6fa45a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f4097740 con 0x7fe6f406c180 2024-09-13T15:43:58.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.739+0000 7fe6fa45a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f40972d0 con 0x7fe6f406b860 2024-09-13T15:43:58.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.740+0000 7fe6f37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe6f406c180 0x7fe6f412d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:58.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.740+0000 7fe6f3fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6f406b860 0x7fe6f406bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:58.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.740+0000 7fe6f89f7640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe6f406c560 0x7fe6f412db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60414/0 (socket says 172.21.15.70:60414) 2024-09-13T15:43:58.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.740+0000 7fe6f89f7640 1 -- 172.21.15.70:0/2941353179 learned_addr learned my addr 172.21.15.70:0/2941353179 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:58.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.740+0000 7fe6f37fe640 1 -- 172.21.15.70:0/2941353179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6f406b860 msgr2=0x7fe6f406bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:58.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.740+0000 7fe6f37fe640 1 --2- 172.21.15.70:0/2941353179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6f406b860 0x7fe6f406bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.740+0000 7fe6f37fe640 1 -- 172.21.15.70:0/2941353179 >> v1:172.21.15.70:6789/0 conn(0x7fe6f406c560 legacy=0x7fe6f412db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:58.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.740+0000 7fe6f37fe640 1 -- 172.21.15.70:0/2941353179 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f406d000 con 0x7fe6f406c180 2024-09-13T15:43:58.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.740+0000 7fe6f2ffd640 1 -- 172.21.15.70:0/2941353179 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092704098 0 0) 0x7fe6f4066060 con 0x7fe6f406c560 2024-09-13T15:43:58.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.740+0000 7fe6f3fff640 1 --2- 172.21.15.70:0/2941353179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6f406b860 0x7fe6f406bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:58.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.740+0000 7fe6f37fe640 1 --2- 172.21.15.70:0/2941353179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f406c180 0x7fe6f412d5f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80097a0 tx=0x7fe6e802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b947185d7007aec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:58.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.741+0000 7fe6f2ffd640 1 -- 172.21.15.70:0/2941353179 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e802adb0 con 0x7fe6f406c180 2024-09-13T15:43:58.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.741+0000 7fe6f2ffd640 1 -- 172.21.15.70:0/2941353179 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e8032440 con 0x7fe6f406c180 2024-09-13T15:43:58.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.741+0000 7fe6f2ffd640 1 -- 172.21.15.70:0/2941353179 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e80327d0 con 0x7fe6f406c180 2024-09-13T15:43:58.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.741+0000 7fe6fa45a640 1 -- 172.21.15.70:0/2941353179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f406c180 msgr2=0x7fe6f412d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:58.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.741+0000 7fe6fa45a640 1 --2- 172.21.15.70:0/2941353179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f406c180 0x7fe6f412d5f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80097a0 tx=0x7fe6e802a060 comp rx=0 tx=0).stop 2024-09-13T15:43:58.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.741+0000 7fe6fa45a640 1 -- 172.21.15.70:0/2941353179 shutdown_connections 2024-09-13T15:43:58.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.741+0000 7fe6fa45a640 1 --2- 172.21.15.70:0/2941353179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f406c180 0x7fe6f412d5f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.741+0000 7fe6fa45a640 1 --2- 172.21.15.70:0/2941353179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6f406b860 0x7fe6f406bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.741+0000 7fe6fa45a640 1 -- 172.21.15.70:0/2941353179 >> 172.21.15.70:0/2941353179 conn(0x7fe6f409e0f0 msgr2=0x7fe6f409e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:58.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.741+0000 7fe6fa45a640 1 -- 172.21.15.70:0/2941353179 shutdown_connections 2024-09-13T15:43:58.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.741+0000 7fe6fa45a640 1 -- 172.21.15.70:0/2941353179 wait complete. 2024-09-13T15:43:58.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.742+0000 7fe6fa45a640 1 Processor -- start 2024-09-13T15:43:58.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.742+0000 7fe6fa45a640 1 -- start start 2024-09-13T15:43:58.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6fa45a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f406b860 0x7fe6f412edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:58.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6fa45a640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe6f406c180 0x7fe6f412f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:58.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6fa45a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6f4130fa0 0x7fe6f412f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:58.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6fa45a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f4131cc0 con 0x7fe6f406b860 2024-09-13T15:43:58.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6fa45a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f4127f30 con 0x7fe6f4130fa0 2024-09-13T15:43:58.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6fa45a640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f4132490 con 0x7fe6f406c180 2024-09-13T15:43:58.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f406b860 0x7fe6f412edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:58.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f89f7640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6f4130fa0 0x7fe6f412f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:58.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f406b860 0x7fe6f412edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40770/0 (socket says 172.21.15.70:40770) 2024-09-13T15:43:58.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f3fff640 1 -- 172.21.15.70:0/1042085677 learned_addr learned my addr 172.21.15.70:0/1042085677 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:58.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f37fe640 1 --2- 172.21.15.70:0/1042085677 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe6f406c180 0x7fe6f412f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:58.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f3fff640 1 -- 172.21.15.70:0/1042085677 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe6f406c180 msgr2=0x7fe6f412f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:58.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f3fff640 1 --2- 172.21.15.70:0/1042085677 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe6f406c180 0x7fe6f412f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f3fff640 1 -- 172.21.15.70:0/1042085677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6f4130fa0 msgr2=0x7fe6f412f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:58.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f3fff640 1 --2- 172.21.15.70:0/1042085677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6f4130fa0 0x7fe6f412f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f3fff640 1 -- 172.21.15.70:0/1042085677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e4009400 con 0x7fe6f406b860 2024-09-13T15:43:58.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f89f7640 1 --2- 172.21.15.70:0/1042085677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6f4130fa0 0x7fe6f412f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:58.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f37fe640 1 --2- 172.21.15.70:0/1042085677 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe6f406c180 0x7fe6f412f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:58.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f3fff640 1 --2- 172.21.15.70:0/1042085677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f406b860 0x7fe6f412edf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400e960 tx=0x7fe6e400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:58.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f0ff9640 1 -- 172.21.15.70:0/1042085677 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400c9a0 con 0x7fe6f406b860 2024-09-13T15:43:58.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f0ff9640 1 -- 172.21.15.70:0/1042085677 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e400cb00 con 0x7fe6f406b860 2024-09-13T15:43:58.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6fa45a640 1 -- 172.21.15.70:0/1042085677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e8009400 con 0x7fe6f406b860 2024-09-13T15:43:58.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6f0ff9640 1 -- 172.21.15.70:0/1042085677 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400ce90 con 0x7fe6f406b860 2024-09-13T15:43:58.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.743+0000 7fe6fa45a640 1 -- 172.21.15.70:0/1042085677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f41353f0 con 0x7fe6f406b860 2024-09-13T15:43:58.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.744+0000 7fe6fa45a640 1 -- 172.21.15.70:0/1042085677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f412dad0 con 0x7fe6f406b860 2024-09-13T15:43:58.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.744+0000 7fe6f0ff9640 1 -- 172.21.15.70:0/1042085677 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fe6e4013470 con 0x7fe6f406b860 2024-09-13T15:43:58.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.745+0000 7fe6f0ff9640 1 -- 172.21.15.70:0/1042085677 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fe6e405cac0 con 0x7fe6f406b860 2024-09-13T15:43:58.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.748+0000 7fe6f0ff9640 1 -- 172.21.15.70:0/1042085677 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fe6e403a6e0 con 0x7fe6f406b860 2024-09-13T15:43:58.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.928+0000 7fe6fa45a640 1 -- 172.21.15.70:0/1042085677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe6f406bc40 con 0x7fe6f406b860 2024-09-13T15:43:58.931 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.929+0000 7fe6f0ff9640 1 -- 172.21.15.70:0/1042085677 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+503 (secure 0 0 0) 0x7fe6e405ce60 con 0x7fe6f406b860 2024-09-13T15:43:58.931 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-13T15:43:58.931 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": { 2024-09-13T15:43:58.932 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.4-850-g50bbe7f3 (50bbe7f395997dcd871813949c49addbbf630361) reef (stable)": 3 2024-09-13T15:43:58.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:58.932 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": { 2024-09-13T15:43:58.932 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.4-850-g50bbe7f3 (50bbe7f395997dcd871813949c49addbbf630361) reef (stable)": 2 2024-09-13T15:43:58.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:58.932 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": { 2024-09-13T15:43:58.932 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.4-850-g50bbe7f3 (50bbe7f395997dcd871813949c49addbbf630361) reef (stable)": 4 2024-09-13T15:43:58.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:58.932 INFO:teuthology.orchestra.run.smithi070.stdout: "overall": { 2024-09-13T15:43:58.932 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph version 18.2.4-850-g50bbe7f3 (50bbe7f395997dcd871813949c49addbbf630361) reef (stable)": 9 2024-09-13T15:43:58.932 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:58.932 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-13T15:43:58.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.931+0000 7fe6fa45a640 1 -- 172.21.15.70:0/1042085677 >> v1:172.21.15.70:6816/1229510400 conn(0x7fe6e0047560 legacy=0x7fe6e0049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:58.933 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.931+0000 7fe6fa45a640 1 -- 172.21.15.70:0/1042085677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f406b860 msgr2=0x7fe6f412edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:58.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.931+0000 7fe6fa45a640 1 --2- 172.21.15.70:0/1042085677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f406b860 0x7fe6f412edf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400e960 tx=0x7fe6e400ee20 comp rx=0 tx=0).stop 2024-09-13T15:43:58.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.931+0000 7fe6fa45a640 1 -- 172.21.15.70:0/1042085677 shutdown_connections 2024-09-13T15:43:58.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.931+0000 7fe6fa45a640 1 --2- 172.21.15.70:0/1042085677 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe6f4130fa0 0x7fe6f412f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.931+0000 7fe6fa45a640 1 --2- 172.21.15.70:0/1042085677 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe6f406c180 0x7fe6f412f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.931+0000 7fe6fa45a640 1 --2- 172.21.15.70:0/1042085677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f406b860 0x7fe6f412edf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:58.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.931+0000 7fe6fa45a640 1 -- 172.21.15.70:0/1042085677 >> 172.21.15.70:0/1042085677 conn(0x7fe6f409e0f0 msgr2=0x7fe6f409b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:58.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.931+0000 7fe6fa45a640 1 -- 172.21.15.70:0/1042085677 shutdown_connections 2024-09-13T15:43:58.934 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:58.931+0000 7fe6fa45a640 1 -- 172.21.15.70:0/1042085677 wait complete. 2024-09-13T15:43:58.943 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd dump -f json-pretty' 2024-09-13T15:43:59.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.035+0000 7f132691f640 1 Processor -- start 2024-09-13T15:43:59.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.035+0000 7f132691f640 1 -- start start 2024-09-13T15:43:59.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.035+0000 7f132691f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f132012a460 0x7f132012a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.035+0000 7f132691f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f132012ad80 0x7f132006c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.035+0000 7f132691f640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f132006c890 0x7f132006ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.038 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.035+0000 7f132691f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f132006d730 con 0x7f132012a460 2024-09-13T15:43:59.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.035+0000 7f132691f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132012a240 con 0x7f132012ad80 2024-09-13T15:43:59.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.035+0000 7f132691f640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13200983c0 con 0x7f132006c890 2024-09-13T15:43:59.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.035+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f132012ad80 0x7f132006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.035+0000 7f131ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f132012a460 0x7f132012a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.036+0000 7f1324ebc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f132006c890 0x7f132006ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.036+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f132012ad80 0x7f132006c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59308/0 (socket says 172.21.15.70:59308) 2024-09-13T15:43:59.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.036+0000 7f131ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f132012a460 0x7f132012a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40786/0 (socket says 172.21.15.70:40786) 2024-09-13T15:43:59.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.036+0000 7f131f7fe640 1 -- 172.21.15.70:0/2441040948 learned_addr learned my addr 172.21.15.70:0/2441040948 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:59.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.036+0000 7f131f7fe640 1 -- 172.21.15.70:0/2441040948 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f132006c890 msgr2=0x7f132006ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:59.039 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.036+0000 7f131f7fe640 1 --2- 172.21.15.70:0/2441040948 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f132006c890 0x7f132006ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.036+0000 7f131f7fe640 1 -- 172.21.15.70:0/2441040948 >> v2:172.21.15.70:3300/0 conn(0x7f132012a460 msgr2=0x7f132012a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:59.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.036+0000 7f131f7fe640 1 --2- 172.21.15.70:0/2441040948 >> v2:172.21.15.70:3300/0 conn(0x7f132012a460 0x7f132012a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.036+0000 7f131f7fe640 1 -- 172.21.15.70:0/2441040948 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132006e730 con 0x7f132012ad80 2024-09-13T15:43:59.040 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.036+0000 7f131ffff640 1 --2- 172.21.15.70:0/2441040948 >> v2:172.21.15.70:3300/0 conn(0x7f132012a460 0x7f132012a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:59.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.036+0000 7f131f7fe640 1 --2- 172.21.15.70:0/2441040948 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f132012ad80 0x7f132006c350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1314009870 tx=0x7f131402a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=892a2c8cbaff1e5a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:59.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.037+0000 7f131effd640 1 -- 172.21.15.70:0/2441040948 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131402adf0 con 0x7f132012ad80 2024-09-13T15:43:59.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.037+0000 7f131effd640 1 -- 172.21.15.70:0/2441040948 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1314032440 con 0x7f132012ad80 2024-09-13T15:43:59.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.037+0000 7f131effd640 1 -- 172.21.15.70:0/2441040948 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13140327d0 con 0x7f132012ad80 2024-09-13T15:43:59.043 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.037+0000 7f132691f640 1 -- 172.21.15.70:0/2441040948 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f132012ad80 msgr2=0x7f132006c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:59.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.037+0000 7f132691f640 1 --2- 172.21.15.70:0/2441040948 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f132012ad80 0x7f132006c350 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1314009870 tx=0x7f131402a0a0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.037+0000 7f132691f640 1 -- 172.21.15.70:0/2441040948 shutdown_connections 2024-09-13T15:43:59.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.037+0000 7f132691f640 1 --2- 172.21.15.70:0/2441040948 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f132006c890 0x7f132006ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.037+0000 7f132691f640 1 --2- 172.21.15.70:0/2441040948 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f132012ad80 0x7f132006c350 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.037+0000 7f132691f640 1 --2- 172.21.15.70:0/2441040948 >> v2:172.21.15.70:3300/0 conn(0x7f132012a460 0x7f132012a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.037+0000 7f132691f640 1 -- 172.21.15.70:0/2441040948 >> 172.21.15.70:0/2441040948 conn(0x7f1320098c40 msgr2=0x7f1320096510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:59.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.037+0000 7f132691f640 1 -- 172.21.15.70:0/2441040948 shutdown_connections 2024-09-13T15:43:59.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.037+0000 7f132691f640 1 -- 172.21.15.70:0/2441040948 wait complete. 2024-09-13T15:43:59.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.038+0000 7f132691f640 1 Processor -- start 2024-09-13T15:43:59.044 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.038+0000 7f132691f640 1 -- start start 2024-09-13T15:43:59.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f132691f640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f132006c890 0x7f13201c2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f132691f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f132012a460 0x7f13201c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f132691f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f132012ad80 0x7f132012e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f132691f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13200711b0 con 0x7f132012ad80 2024-09-13T15:43:59.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f132691f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13201303b0 con 0x7f132012a460 2024-09-13T15:43:59.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f132691f640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1320130570 con 0x7f132006c890 2024-09-13T15:43:59.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f131ffff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f132006c890 0x7f13201c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f132012a460 0x7f13201c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f132012a460 0x7f13201c2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59320/0 (socket says 172.21.15.70:59320) 2024-09-13T15:43:59.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f1324ebc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f132012ad80 0x7f132012e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.045 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f131f7fe640 1 -- 172.21.15.70:0/1455222776 learned_addr learned my addr 172.21.15.70:0/1455222776 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:59.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f131ffff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f132006c890 0x7f13201c2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58528/0 (socket says 172.21.15.70:58528) 2024-09-13T15:43:59.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f131f7fe640 1 -- 172.21.15.70:0/1455222776 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f132006c890 msgr2=0x7f13201c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:59.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f131f7fe640 1 --2- 172.21.15.70:0/1455222776 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f132006c890 0x7f13201c2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f131f7fe640 1 -- 172.21.15.70:0/1455222776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f132012ad80 msgr2=0x7f132012e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:59.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f131f7fe640 1 --2- 172.21.15.70:0/1455222776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f132012ad80 0x7f132012e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f131f7fe640 1 -- 172.21.15.70:0/1455222776 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1310009400 con 0x7f132012a460 2024-09-13T15:43:59.046 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f131f7fe640 1 --2- 172.21.15.70:0/1455222776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f132012a460 0x7f13201c2e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1314000c00 tx=0x7f13140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:59.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f131cff9640 1 -- 172.21.15.70:0/1455222776 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131403d830 con 0x7f132012a460 2024-09-13T15:43:59.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f131cff9640 1 -- 172.21.15.70:0/1455222776 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131403d990 con 0x7f132012a460 2024-09-13T15:43:59.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f132691f640 1 -- 172.21.15.70:0/1455222776 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13140094d0 con 0x7f132012a460 2024-09-13T15:43:59.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f131cff9640 1 -- 172.21.15.70:0/1455222776 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131403dd40 con 0x7f132012a460 2024-09-13T15:43:59.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.039+0000 7f132691f640 1 -- 172.21.15.70:0/1455222776 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1320130c60 con 0x7f132012a460 2024-09-13T15:43:59.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.040+0000 7f131cff9640 1 -- 172.21.15.70:0/1455222776 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f1314039040 con 0x7f132012a460 2024-09-13T15:43:59.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.040+0000 7f132691f640 1 -- 172.21.15.70:0/1455222776 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f132006bb50 con 0x7f132012a460 2024-09-13T15:43:59.047 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.041+0000 7f131cff9640 1 -- 172.21.15.70:0/1455222776 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f131407d400 con 0x7f132012a460 2024-09-13T15:43:59.048 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.044+0000 7f131cff9640 1 -- 172.21.15.70:0/1455222776 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f13140474b0 con 0x7f132012a460 2024-09-13T15:43:59.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.179+0000 7f132691f640 1 -- 172.21.15.70:0/1455222776 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f1320130f10 con 0x7f132012a460 2024-09-13T15:43:59.183 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.180+0000 7f131cff9640 1 -- 172.21.15.70:0/1455222776 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v29) v1 ==== 81+0+16651 (secure 0 0 0) 0x7f1314037030 con 0x7f132012a460 2024-09-13T15:43:59.184 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:43:59.184 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-13T15:43:59.184 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 29, 2024-09-13T15:43:59.184 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "45bbcdad-f4fb-4639-ab42-ccda8de151dd", 2024-09-13T15:43:59.184 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2024-09-13T15:42:09.863982+0000", 2024-09-13T15:43:59.185 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2024-09-13T15:43:56.576725+0000", 2024-09-13T15:43:59.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2024-09-13T15:43:55.573914+0000", 2024-09-13T15:43:59.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2024-09-13T15:42:11.714506+0000", 2024-09-13T15:43:59.185 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T15:43:59.185 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2024-09-13T15:43:59.185 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2024-09-13T15:43:59.185 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2024-09-13T15:43:59.185 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2024-09-13T15:43:59.186 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2024-09-13T15:43:59.186 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2024-09-13T15:43:59.186 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-13T15:43:59.186 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 8, 2024-09-13T15:43:59.186 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T15:43:59.186 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T15:43:59.186 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T15:43:59.187 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2024-09-13T15:43:59.187 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 2, 2024-09-13T15:43:59.187 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 4, 2024-09-13T15:43:59.187 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "quincy", 2024-09-13T15:43:59.187 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2024-09-13T15:43:59.187 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "quincy", 2024-09-13T15:43:59.187 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2024-09-13T15:43:59.188 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2024-09-13T15:43:59.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.188 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2024-09-13T15:43:59.188 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2024-09-13T15:43:59.188 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-13T15:42:21.000499+0000", 2024-09-13T15:43:59.188 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-13T15:43:59.188 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-13T15:43:59.189 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-13T15:43:59.189 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2024-09-13T15:43:59.189 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2024-09-13T15:43:59.189 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-13T15:43:59.189 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-13T15:43:59.189 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-13T15:43:59.189 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T15:43:59.190 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T15:43:59.190 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-13T15:43:59.190 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-13T15:43:59.190 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2024-09-13T15:43:59.190 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2024-09-13T15:43:59.190 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2024-09-13T15:43:59.190 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2024-09-13T15:43:59.190 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2024-09-13T15:43:59.191 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-13T15:43:59.191 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-13T15:43:59.191 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-13T15:43:59.191 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-13T15:43:59.191 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-13T15:43:59.191 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-13T15:43:59.192 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-13T15:43:59.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.192 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "12", 2024-09-13T15:43:59.192 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-13T15:43:59.192 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T15:43:59.192 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T15:43:59.192 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-13T15:43:59.192 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-13T15:43:59.193 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-13T15:43:59.193 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-13T15:43:59.193 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-13T15:43:59.193 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-13T15:43:59.193 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-13T15:43:59.193 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-13T15:43:59.193 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-13T15:43:59.194 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-13T15:43:59.194 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-13T15:43:59.194 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-13T15:43:59.194 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-13T15:43:59.194 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-13T15:43:59.194 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-13T15:43:59.194 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T15:43:59.195 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T15:43:59.195 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T15:43:59.195 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-13T15:43:59.195 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-13T15:43:59.195 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-13T15:43:59.195 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-13T15:43:59.195 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-13T15:43:59.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.196 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-13T15:43:59.196 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-13T15:43:59.196 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-13T15:43:59.196 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-13T15:43:59.196 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-13T15:43:59.196 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T15:43:59.197 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-13T15:43:59.197 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-13T15:43:59.197 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-13T15:43:59.197 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-13T15:43:59.197 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-13T15:43:59.197 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-13T15:43:59.197 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2024-09-13T15:43:59.197 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2024-09-13T15:43:59.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.198 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-13T15:43:59.198 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2024-09-13T15:43:59.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.198 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-13T15:43:59.198 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4, 2024-09-13T15:43:59.198 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4, 2024-09-13T15:43:59.199 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.5, 2024-09-13T15:43:59.199 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-13T15:43:59.199 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-13T15:43:59.199 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-13T15:43:59.199 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-13T15:43:59.199 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-13T15:43:59.199 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.200 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2024-09-13T15:43:59.200 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2024-09-13T15:43:59.200 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-13T15:42:26.149909+0000", 2024-09-13T15:43:59.200 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-13T15:43:59.200 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T15:43:59.201 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-13T15:43:59.201 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2024-09-13T15:43:59.201 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2024-09-13T15:43:59.201 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-13T15:43:59.201 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-13T15:43:59.201 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-13T15:43:59.201 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T15:43:59.202 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T15:43:59.202 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-13T15:43:59.202 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-13T15:43:59.202 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2024-09-13T15:43:59.203 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2024-09-13T15:43:59.203 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2024-09-13T15:43:59.203 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2024-09-13T15:43:59.203 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2024-09-13T15:43:59.203 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-13T15:43:59.203 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-13T15:43:59.203 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-13T15:43:59.204 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-13T15:43:59.204 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-13T15:43:59.204 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-13T15:43:59.204 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-13T15:43:59.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.204 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "16", 2024-09-13T15:43:59.204 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-13T15:43:59.205 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T15:43:59.205 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T15:43:59.205 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-13T15:43:59.205 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-13T15:43:59.205 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 3, 2024-09-13T15:43:59.205 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 16, 2024-09-13T15:43:59.205 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-13T15:43:59.206 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-13T15:43:59.206 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-13T15:43:59.206 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-13T15:43:59.206 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-13T15:43:59.206 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-13T15:43:59.206 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-13T15:43:59.206 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-13T15:43:59.206 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-13T15:43:59.207 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-13T15:43:59.207 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-13T15:43:59.207 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T15:43:59.207 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T15:43:59.207 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T15:43:59.207 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-13T15:43:59.208 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-13T15:43:59.208 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-13T15:43:59.208 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-13T15:43:59.208 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-13T15:43:59.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.208 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-13T15:43:59.208 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-13T15:43:59.208 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-13T15:43:59.209 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-13T15:43:59.209 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-13T15:43:59.209 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T15:43:59.209 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-13T15:43:59.209 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-13T15:43:59.209 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-13T15:43:59.209 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-13T15:43:59.210 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-13T15:43:59.210 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-13T15:43:59.210 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-13T15:43:59.210 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-13T15:43:59.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.210 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-13T15:43:59.210 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.5, 2024-09-13T15:43:59.211 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.5, 2024-09-13T15:43:59.211 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-13T15:43:59.211 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2.5, 2024-09-13T15:43:59.211 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.5, 2024-09-13T15:43:59.211 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-13T15:43:59.211 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-13T15:43:59.211 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-13T15:43:59.212 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.212 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.212 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-13T15:43:59.212 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-13T15:43:59.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.212 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-13T15:43:59.212 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "265c6f1d-08a7-4c43-b2a6-2417ab814825", 2024-09-13T15:43:59.213 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-13T15:43:59.213 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-13T15:43:59.213 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-13T15:43:59.213 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-13T15:43:59.213 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 9, 2024-09-13T15:43:59.213 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 19, 2024-09-13T15:43:59.213 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 25, 2024-09-13T15:43:59.214 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 25, 2024-09-13T15:43:59.214 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 20, 2024-09-13T15:43:59.214 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-13T15:43:59.214 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-13T15:43:59.214 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.214 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.214 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2024-09-13T15:43:59.215 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2246103037 2024-09-13T15:43:59.215 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.215 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.216 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-13T15:43:59.216 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.216 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.216 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2024-09-13T15:43:59.216 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2246103037 2024-09-13T15:43:59.216 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.217 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.217 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-13T15:43:59.217 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.217 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.217 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2024-09-13T15:43:59.218 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2246103037 2024-09-13T15:43:59.218 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.218 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.218 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-13T15:43:59.218 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.218 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.218 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2024-09-13T15:43:59.219 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2246103037 2024-09-13T15:43:59.219 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.219 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.219 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6800/2246103037", 2024-09-13T15:43:59.219 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6801/2246103037", 2024-09-13T15:43:59.219 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6803/2246103037", 2024-09-13T15:43:59.219 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6802/2246103037", 2024-09-13T15:43:59.219 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-13T15:43:59.219 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-13T15:43:59.219 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-13T15:43:59.219 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.220 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-13T15:43:59.220 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "ba4d90f9-6972-45df-800f-92ad808b06ec", 2024-09-13T15:43:59.220 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-13T15:43:59.220 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-13T15:43:59.220 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-13T15:43:59.220 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-13T15:43:59.220 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 9, 2024-09-13T15:43:59.220 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 20, 2024-09-13T15:43:59.220 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 26, 2024-09-13T15:43:59.220 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 26, 2024-09-13T15:43:59.220 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 21, 2024-09-13T15:43:59.220 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-13T15:43:59.220 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-13T15:43:59.221 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.221 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.221 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2024-09-13T15:43:59.221 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 853418053 2024-09-13T15:43:59.221 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.221 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.221 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-13T15:43:59.221 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.221 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.221 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2024-09-13T15:43:59.222 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 853418053 2024-09-13T15:43:59.222 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.222 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.222 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-13T15:43:59.222 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.222 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.222 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2024-09-13T15:43:59.222 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 853418053 2024-09-13T15:43:59.222 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.222 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.222 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-13T15:43:59.223 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.223 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.223 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2024-09-13T15:43:59.223 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 853418053 2024-09-13T15:43:59.223 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.223 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.223 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6804/853418053", 2024-09-13T15:43:59.223 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6805/853418053", 2024-09-13T15:43:59.223 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/853418053", 2024-09-13T15:43:59.223 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6806/853418053", 2024-09-13T15:43:59.223 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-13T15:43:59.224 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-13T15:43:59.224 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-13T15:43:59.224 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.224 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-13T15:43:59.224 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "b74d8eb5-3082-4d69-a755-7f5f92dec5b1", 2024-09-13T15:43:59.224 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-13T15:43:59.224 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-13T15:43:59.224 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-13T15:43:59.224 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-13T15:43:59.224 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 9, 2024-09-13T15:43:59.224 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 21, 2024-09-13T15:43:59.225 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 27, 2024-09-13T15:43:59.225 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-13T15:43:59.225 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 22, 2024-09-13T15:43:59.225 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-13T15:43:59.225 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-13T15:43:59.225 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.225 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.225 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2024-09-13T15:43:59.225 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2115999109 2024-09-13T15:43:59.225 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.225 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.226 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-13T15:43:59.226 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.226 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.226 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2024-09-13T15:43:59.226 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2115999109 2024-09-13T15:43:59.226 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.226 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.226 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-13T15:43:59.226 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.226 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.227 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2024-09-13T15:43:59.227 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2115999109 2024-09-13T15:43:59.227 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.227 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.227 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-13T15:43:59.227 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.227 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.227 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2024-09-13T15:43:59.227 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2115999109 2024-09-13T15:43:59.227 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.227 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.228 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6808/2115999109", 2024-09-13T15:43:59.228 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6809/2115999109", 2024-09-13T15:43:59.228 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6811/2115999109", 2024-09-13T15:43:59.228 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6810/2115999109", 2024-09-13T15:43:59.228 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-13T15:43:59.228 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-13T15:43:59.228 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-13T15:43:59.228 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.228 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-13T15:43:59.228 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "02835b7d-e2cf-4ef8-87ba-b225593689d3", 2024-09-13T15:43:59.228 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-13T15:43:59.229 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-13T15:43:59.229 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-13T15:43:59.229 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-13T15:43:59.229 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 9, 2024-09-13T15:43:59.229 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 22, 2024-09-13T15:43:59.229 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 28, 2024-09-13T15:43:59.229 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 28, 2024-09-13T15:43:59.229 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 23, 2024-09-13T15:43:59.229 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-13T15:43:59.229 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-13T15:43:59.230 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.230 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.230 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2024-09-13T15:43:59.230 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2095410362 2024-09-13T15:43:59.230 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.230 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.230 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-13T15:43:59.231 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.231 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.231 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2024-09-13T15:43:59.231 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2095410362 2024-09-13T15:43:59.231 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.231 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.231 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-13T15:43:59.232 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.232 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.232 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2024-09-13T15:43:59.232 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2095410362 2024-09-13T15:43:59.232 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.232 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.232 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-13T15:43:59.232 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-13T15:43:59.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.233 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-13T15:43:59.233 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2024-09-13T15:43:59.233 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2095410362 2024-09-13T15:43:59.233 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.233 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.233 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6812/2095410362", 2024-09-13T15:43:59.233 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6813/2095410362", 2024-09-13T15:43:59.233 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6815/2095410362", 2024-09-13T15:43:59.233 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6814/2095410362", 2024-09-13T15:43:59.233 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-13T15:43:59.233 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-13T15:43:59.234 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-13T15:43:59.234 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-13T15:43:59.234 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.234 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-13T15:43:59.234 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2024-09-13T15:43:59.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.234 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-13T15:43:59.234 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-13T15:43:59.234 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-13T15:43:59.234 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-13T15:43:59.235 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2024-09-13T15:43:59.235 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-13T15:43:59.235 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-13T15:42:16.853420+0000", 2024-09-13T15:43:59.235 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 20 2024-09-13T15:43:59.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.235 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-13T15:43:59.235 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-13T15:43:59.235 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-13T15:43:59.235 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-13T15:43:59.236 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2024-09-13T15:43:59.236 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-13T15:43:59.236 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-13T15:42:16.807491+0000", 2024-09-13T15:43:59.236 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 21 2024-09-13T15:43:59.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.236 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-13T15:43:59.236 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-13T15:43:59.236 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-13T15:43:59.236 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-13T15:43:59.236 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2024-09-13T15:43:59.237 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-13T15:43:59.237 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-13T15:42:16.925567+0000", 2024-09-13T15:43:59.237 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 22 2024-09-13T15:43:59.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-13T15:43:59.237 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-13T15:43:59.237 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "2024-09-13T15:43:48.549448+0000", 2024-09-13T15:43:59.237 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-13T15:43:59.237 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-13T15:43:59.237 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2024-09-13T15:43:59.238 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-13T15:43:59.238 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-13T15:42:16.883852+0000", 2024-09-13T15:43:59.238 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-13T15:43:59.238 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.238 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-13T15:43:59.238 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2024-09-13T15:43:59.238 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2024-09-13T15:43:59.238 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2024-09-13T15:43:59.238 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2024-09-13T15:43:59.238 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2024-09-13T15:43:59.238 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": { 2024-09-13T15:43:59.238 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6816/32242": "2024-09-14T15:43:41.526957+0000", 2024-09-13T15:43:59.239 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/2914106548": "2024-09-14T15:43:41.526957+0000", 2024-09-13T15:43:59.239 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3671770748": "2024-09-14T15:43:41.526957+0000", 2024-09-13T15:43:59.239 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3345804336": "2024-09-14T15:43:41.526957+0000", 2024-09-13T15:43:59.239 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3873928986": "2024-09-14T15:43:41.526957+0000" 2024-09-13T15:43:59.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.239 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2024-09-13T15:43:59.239 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2024-09-13T15:43:59.239 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2024-09-13T15:43:59.239 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2024-09-13T15:43:59.240 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2024-09-13T15:43:59.240 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2024-09-13T15:43:59.240 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2024-09-13T15:43:59.240 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2024-09-13T15:43:59.240 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-13T15:43:59.240 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2024-09-13T15:43:59.240 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2024-09-13T15:43:59.240 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2024-09-13T15:43:59.241 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2024-09-13T15:43:59.241 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2024-09-13T15:43:59.241 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2024-09-13T15:43:59.241 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2024-09-13T15:43:59.241 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2024-09-13T15:43:59.241 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2024-09-13T15:43:59.241 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2024-09-13T15:43:59.241 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2024-09-13T15:43:59.241 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-13T15:43:59.241 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-13T15:43:59.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.182+0000 7f132691f640 1 -- 172.21.15.70:0/1455222776 >> v1:172.21.15.70:6816/1229510400 conn(0x7f130c047560 legacy=0x7f130c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:59.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.182+0000 7f132691f640 1 -- 172.21.15.70:0/1455222776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f132012a460 msgr2=0x7f13201c2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:59.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.182+0000 7f132691f640 1 --2- 172.21.15.70:0/1455222776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f132012a460 0x7f13201c2e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1314000c00 tx=0x7f13140046c0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.182+0000 7f132691f640 1 -- 172.21.15.70:0/1455222776 shutdown_connections 2024-09-13T15:43:59.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.182+0000 7f132691f640 1 --2- 172.21.15.70:0/1455222776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f132012ad80 0x7f132012e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.182+0000 7f132691f640 1 --2- 172.21.15.70:0/1455222776 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f132012a460 0x7f13201c2e70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.182+0000 7f132691f640 1 --2- 172.21.15.70:0/1455222776 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f132006c890 0x7f13201c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.182+0000 7f132691f640 1 -- 172.21.15.70:0/1455222776 >> 172.21.15.70:0/1455222776 conn(0x7f1320098c40 msgr2=0x7f132006fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:59.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.182+0000 7f132691f640 1 -- 172.21.15.70:0/1455222776 shutdown_connections 2024-09-13T15:43:59.244 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.182+0000 7f132691f640 1 -- 172.21.15.70:0/1455222776 wait complete. 2024-09-13T15:43:59.245 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs dump' 2024-09-13T15:43:59.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.337+0000 7fedce145640 1 Processor -- start 2024-09-13T15:43:59.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.337+0000 7fedce145640 1 -- start start 2024-09-13T15:43:59.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.337+0000 7fedce145640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fedc8129f40 0x7fedc812a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.337+0000 7fedce145640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fedc812a860 0x7fedc806b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.337+0000 7fedce145640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc806de60 con 0x7fedc806b780 2024-09-13T15:43:59.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.337+0000 7fedce145640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc806e230 con 0x7fedc8129f40 2024-09-13T15:43:59.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.337+0000 7fedce145640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedc806cd40 con 0x7fedc812a860 2024-09-13T15:43:59.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.337+0000 7fedbffff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fedc812a860 0x7fedc806b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.337+0000 7fedc77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fedc8129f40 0x7fedc812a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.337+0000 7fedc7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fedc806b780 0x7fedc806bbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60422/0 (socket says 172.21.15.70:60422) 2024-09-13T15:43:59.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.337+0000 7fedc7fff640 1 -- 172.21.15.70:0/3630336250 learned_addr learned my addr 172.21.15.70:0/3630336250 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:59.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.337+0000 7fedc77fe640 1 --2- 172.21.15.70:0/3630336250 >> v2:172.21.15.70:3300/0 conn(0x7fedc8129f40 0x7fedc812a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40802/0 (socket says 172.21.15.70:40802) 2024-09-13T15:43:59.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.337+0000 7fedbffff640 1 --2- 172.21.15.70:0/3630336250 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fedc812a860 0x7fedc806b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58532/0 (socket says 172.21.15.70:58532) 2024-09-13T15:43:59.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.338+0000 7fedbffff640 1 -- 172.21.15.70:0/3630336250 >> v2:172.21.15.70:3300/0 conn(0x7fedc8129f40 msgr2=0x7fedc812a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:59.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.338+0000 7fedbffff640 1 --2- 172.21.15.70:0/3630336250 >> v2:172.21.15.70:3300/0 conn(0x7fedc8129f40 0x7fedc812a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.338+0000 7fedbffff640 1 -- 172.21.15.70:0/3630336250 >> v1:172.21.15.70:6789/0 conn(0x7fedc806b780 legacy=0x7fedc806bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:59.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.338+0000 7fedbffff640 1 -- 172.21.15.70:0/3630336250 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc806d680 con 0x7fedc812a860 2024-09-13T15:43:59.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.338+0000 7fedc6ffd640 1 -- 172.21.15.70:0/3630336250 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826946442 0 0) 0x7fedc806de60 con 0x7fedc806b780 2024-09-13T15:43:59.342 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.338+0000 7fedc77fe640 1 --2- 172.21.15.70:0/3630336250 >> v2:172.21.15.70:3300/0 conn(0x7fedc8129f40 0x7fedc812a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:59.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.338+0000 7fedbffff640 1 --2- 172.21.15.70:0/3630336250 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fedc812a860 0x7fedc806b240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fedb00097a0 tx=0x7fedb002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4478dd38a42528aa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:59.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.338+0000 7fedc6ffd640 1 -- 172.21.15.70:0/3630336250 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb002ae50 con 0x7fedc812a860 2024-09-13T15:43:59.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.339+0000 7fedc6ffd640 1 -- 172.21.15.70:0/3630336250 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb0032440 con 0x7fedc812a860 2024-09-13T15:43:59.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.339+0000 7fedc6ffd640 1 -- 172.21.15.70:0/3630336250 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00327d0 con 0x7fedc812a860 2024-09-13T15:43:59.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.339+0000 7fedce145640 1 -- 172.21.15.70:0/3630336250 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fedc812a860 msgr2=0x7fedc806b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:59.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.339+0000 7fedce145640 1 --2- 172.21.15.70:0/3630336250 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fedc812a860 0x7fedc806b240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fedb00097a0 tx=0x7fedb002a100 comp rx=0 tx=0).stop 2024-09-13T15:43:59.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.339+0000 7fedce145640 1 -- 172.21.15.70:0/3630336250 shutdown_connections 2024-09-13T15:43:59.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.339+0000 7fedce145640 1 --2- 172.21.15.70:0/3630336250 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fedc812a860 0x7fedc806b240 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.339+0000 7fedce145640 1 --2- 172.21.15.70:0/3630336250 >> v2:172.21.15.70:3300/0 conn(0x7fedc8129f40 0x7fedc812a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.339+0000 7fedce145640 1 -- 172.21.15.70:0/3630336250 >> 172.21.15.70:0/3630336250 conn(0x7fedc8098c40 msgr2=0x7fedc8096510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:59.344 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.339+0000 7fedce145640 1 -- 172.21.15.70:0/3630336250 shutdown_connections 2024-09-13T15:43:59.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.339+0000 7fedce145640 1 -- 172.21.15.70:0/3630336250 wait complete. 2024-09-13T15:43:59.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.340+0000 7fedce145640 1 Processor -- start 2024-09-13T15:43:59.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.340+0000 7fedce145640 1 -- start start 2024-09-13T15:43:59.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.340+0000 7fedce145640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedc806b780 0x7fedc8120e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.340+0000 7fedce145640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fedc8129f40 0x7fedc8121350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.345 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.340+0000 7fedce145640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fedc812a860 0x7fedc8121890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.340+0000 7fedce145640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8121fe0 con 0x7fedc806b780 2024-09-13T15:43:59.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.340+0000 7fedce145640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8127490 con 0x7fedc812a860 2024-09-13T15:43:59.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.340+0000 7fedce145640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8133220 con 0x7fedc8129f40 2024-09-13T15:43:59.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.340+0000 7fedc77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedc806b780 0x7fedc8120e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedc77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedc806b780 0x7fedc8120e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40818/0 (socket says 172.21.15.70:40818) 2024-09-13T15:43:59.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedc77fe640 1 -- 172.21.15.70:0/3796479520 learned_addr learned my addr 172.21.15.70:0/3796479520 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:59.347 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedbffff640 1 --2- 172.21.15.70:0/3796479520 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fedc8129f40 0x7fedc8121350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedc7fff640 1 --2- 172.21.15.70:0/3796479520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fedc812a860 0x7fedc8121890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedc77fe640 1 -- 172.21.15.70:0/3796479520 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fedc8129f40 msgr2=0x7fedc8121350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:59.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedc77fe640 1 --2- 172.21.15.70:0/3796479520 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fedc8129f40 0x7fedc8121350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedc77fe640 1 -- 172.21.15.70:0/3796479520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fedc812a860 msgr2=0x7fedc8121890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:59.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedc77fe640 1 --2- 172.21.15.70:0/3796479520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fedc812a860 0x7fedc8121890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedc77fe640 1 -- 172.21.15.70:0/3796479520 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb8009400 con 0x7fedc806b780 2024-09-13T15:43:59.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedbffff640 1 --2- 172.21.15.70:0/3796479520 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fedc8129f40 0x7fedc8121350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:59.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedc77fe640 1 --2- 172.21.15.70:0/3796479520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedc806b780 0x7fedc8120e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fedb800ec10 tx=0x7fedb800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:59.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedc4ff9640 1 -- 172.21.15.70:0/3796479520 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8009760 con 0x7fedc806b780 2024-09-13T15:43:59.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedc4ff9640 1 -- 172.21.15.70:0/3796479520 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb800cc30 con 0x7fedc806b780 2024-09-13T15:43:59.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedce145640 1 -- 172.21.15.70:0/3796479520 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb0009400 con 0x7fedc806b780 2024-09-13T15:43:59.348 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedc4ff9640 1 -- 172.21.15.70:0/3796479520 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8013470 con 0x7fedc806b780 2024-09-13T15:43:59.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.341+0000 7fedce145640 1 -- 172.21.15.70:0/3796479520 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc81337f0 con 0x7fedc806b780 2024-09-13T15:43:59.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.342+0000 7fedc4ff9640 1 -- 172.21.15.70:0/3796479520 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fedb80135d0 con 0x7fedc806b780 2024-09-13T15:43:59.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.342+0000 7fedce145640 1 -- 172.21.15.70:0/3796479520 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc80981e0 con 0x7fedc806b780 2024-09-13T15:43:59.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.343+0000 7fedc4ff9640 1 -- 172.21.15.70:0/3796479520 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3023+0+0 (secure 0 0 0) 0x7fedb805cbd0 con 0x7fedc806b780 2024-09-13T15:43:59.349 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.346+0000 7fedc4ff9640 1 -- 172.21.15.70:0/3796479520 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fedb803a7f0 con 0x7fedc806b780 2024-09-13T15:43:59.503 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.501+0000 7fedce145640 1 -- 172.21.15.70:0/3796479520 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fedc8072360 con 0x7fedc806b780 2024-09-13T15:43:59.503 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.501+0000 7fedc4ff9640 1 -- 172.21.15.70:0/3796479520 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 1 v1) v1 ==== 75+0+358 (secure 0 0 0) 0x7fedb8017030 con 0x7fedc806b780 2024-09-13T15:43:59.504 INFO:teuthology.orchestra.run.smithi070.stdout:e1 2024-09-13T15:43:59.504 INFO:teuthology.orchestra.run.smithi070.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-13T15:43:59.504 INFO:teuthology.orchestra.run.smithi070.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-13T15:43:59.504 INFO:teuthology.orchestra.run.smithi070.stdout:legacy client fscid: -1 2024-09-13T15:43:59.504 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:43:59.504 INFO:teuthology.orchestra.run.smithi070.stdout:No filesystems configured 2024-09-13T15:43:59.506 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 1 2024-09-13T15:43:59.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.503+0000 7fedce145640 1 -- 172.21.15.70:0/3796479520 >> v1:172.21.15.70:6816/1229510400 conn(0x7feda8047560 legacy=0x7feda8049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:59.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.503+0000 7fedce145640 1 -- 172.21.15.70:0/3796479520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedc806b780 msgr2=0x7fedc8120e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:59.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.503+0000 7fedce145640 1 --2- 172.21.15.70:0/3796479520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedc806b780 0x7fedc8120e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fedb800ec10 tx=0x7fedb800c520 comp rx=0 tx=0).stop 2024-09-13T15:43:59.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.503+0000 7fedce145640 1 -- 172.21.15.70:0/3796479520 shutdown_connections 2024-09-13T15:43:59.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.503+0000 7fedce145640 1 --2- 172.21.15.70:0/3796479520 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fedc812a860 0x7fedc8121890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.503+0000 7fedce145640 1 --2- 172.21.15.70:0/3796479520 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fedc8129f40 0x7fedc8121350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.503+0000 7fedce145640 1 --2- 172.21.15.70:0/3796479520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedc806b780 0x7fedc8120e10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.503+0000 7fedce145640 1 -- 172.21.15.70:0/3796479520 >> 172.21.15.70:0/3796479520 conn(0x7fedc8098c40 msgr2=0x7fedc806e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:59.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.503+0000 7fedce145640 1 -- 172.21.15.70:0/3796479520 shutdown_connections 2024-09-13T15:43:59.506 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.503+0000 7fedce145640 1 -- 172.21.15.70:0/3796479520 wait complete. 2024-09-13T15:43:59.516 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd require-osd-release reef' 2024-09-13T15:43:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.658+0000 7f154e533640 1 Processor -- start 2024-09-13T15:43:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.658+0000 7f154e533640 1 -- start start 2024-09-13T15:43:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.658+0000 7f154e533640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f154812b770 0x7f154812bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.658+0000 7f154e533640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f154806b240 0x7f154806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.658+0000 7f154e533640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154812c090 con 0x7f154806bb60 2024-09-13T15:43:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.658+0000 7f154e533640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1548097ea0 con 0x7f154812b770 2024-09-13T15:43:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.658+0000 7f154e533640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f154812b2b0 con 0x7f154806b240 2024-09-13T15:43:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.658+0000 7f1547fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f154812b770 0x7f154812bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.658+0000 7f154cad0640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f154806bb60 0x7f1548132530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60426/0 (socket says 172.21.15.70:60426) 2024-09-13T15:43:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.658+0000 7f154cad0640 1 -- 172.21.15.70:0/1237719788 learned_addr learned my addr 172.21.15.70:0/1237719788 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:43:59.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.658+0000 7f15477fe640 1 --2- 172.21.15.70:0/1237719788 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f154806b240 0x7f154806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.659+0000 7f1546ffd640 1 -- 172.21.15.70:0/1237719788 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358130720 0 0) 0x7f154812c090 con 0x7f154806bb60 2024-09-13T15:43:59.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.659+0000 7f1547fff640 1 -- 172.21.15.70:0/1237719788 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f154806b240 msgr2=0x7f154806b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:43:59.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.659+0000 7f1547fff640 1 --2- 172.21.15.70:0/1237719788 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f154806b240 0x7f154806b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.659+0000 7f1547fff640 1 -- 172.21.15.70:0/1237719788 >> v1:172.21.15.70:6789/0 conn(0x7f154806bb60 legacy=0x7f1548132530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:59.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.659+0000 7f1547fff640 1 -- 172.21.15.70:0/1237719788 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154806bfd0 con 0x7f154812b770 2024-09-13T15:43:59.662 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.659+0000 7f15477fe640 1 --2- 172.21.15.70:0/1237719788 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f154806b240 0x7f154806b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:43:59.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.659+0000 7f1547fff640 1 --2- 172.21.15.70:0/1237719788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f154812b770 0x7f154812bb50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f15380097d0 tx=0x7f153802a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec31841c1cee4881 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:59.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.659+0000 7f1546ffd640 1 -- 172.21.15.70:0/1237719788 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153802ae20 con 0x7f154812b770 2024-09-13T15:43:59.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.659+0000 7f1546ffd640 1 -- 172.21.15.70:0/1237719788 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1538032440 con 0x7f154812b770 2024-09-13T15:43:59.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.659+0000 7f1546ffd640 1 -- 172.21.15.70:0/1237719788 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15380327d0 con 0x7f154812b770 2024-09-13T15:43:59.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.660+0000 7f154e533640 1 -- 172.21.15.70:0/1237719788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f154812b770 msgr2=0x7f154812bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:59.667 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.660+0000 7f154e533640 1 --2- 172.21.15.70:0/1237719788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f154812b770 0x7f154812bb50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f15380097d0 tx=0x7f153802a0d0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.660+0000 7f154e533640 1 -- 172.21.15.70:0/1237719788 shutdown_connections 2024-09-13T15:43:59.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.660+0000 7f154e533640 1 --2- 172.21.15.70:0/1237719788 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f154806b240 0x7f154806b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.660+0000 7f154e533640 1 --2- 172.21.15.70:0/1237719788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f154812b770 0x7f154812bb50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.660+0000 7f154e533640 1 -- 172.21.15.70:0/1237719788 >> 172.21.15.70:0/1237719788 conn(0x7f1548098c40 msgr2=0x7f1548096510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:43:59.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.660+0000 7f154e533640 1 -- 172.21.15.70:0/1237719788 shutdown_connections 2024-09-13T15:43:59.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.660+0000 7f154e533640 1 -- 172.21.15.70:0/1237719788 wait complete. 2024-09-13T15:43:59.668 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.661+0000 7f154e533640 1 Processor -- start 2024-09-13T15:43:59.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.661+0000 7f154e533640 1 -- start start 2024-09-13T15:43:59.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.661+0000 7f154e533640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f154806b240 0x7f154808e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.661+0000 7f154e533640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f154806bb60 0x7f154808e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.661+0000 7f154e533640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f154812b770 0x7f15481bff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:43:59.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.661+0000 7f154e533640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15480917e0 con 0x7f154806bb60 2024-09-13T15:43:59.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.661+0000 7f154e533640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154808f630 con 0x7f154812b770 2024-09-13T15:43:59.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.661+0000 7f154e533640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1548090f80 con 0x7f154806b240 2024-09-13T15:43:59.669 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.661+0000 7f1547fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f154806b240 0x7f154808e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.662+0000 7f154cad0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f154812b770 0x7f15481bff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.662+0000 7f1547fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f154806b240 0x7f154808e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58556/0 (socket says 172.21.15.70:58556) 2024-09-13T15:43:59.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.662+0000 7f1547fff640 1 -- 172.21.15.70:0/2001652876 learned_addr learned my addr 172.21.15.70:0/2001652876 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:43:59.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.662+0000 7f15477fe640 1 --2- 172.21.15.70:0/2001652876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f154806bb60 0x7f154808e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:43:59.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.662+0000 7f1547fff640 1 -- 172.21.15.70:0/2001652876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f154812b770 msgr2=0x7f15481bff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:59.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.662+0000 7f1547fff640 1 --2- 172.21.15.70:0/2001652876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f154812b770 0x7f15481bff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.670 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.662+0000 7f1547fff640 1 -- 172.21.15.70:0/2001652876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f154806bb60 msgr2=0x7f154808e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:43:59.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.662+0000 7f1547fff640 1 --2- 172.21.15.70:0/2001652876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f154806bb60 0x7f154808e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:43:59.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.662+0000 7f1547fff640 1 -- 172.21.15.70:0/2001652876 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c009400 con 0x7f154806b240 2024-09-13T15:43:59.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.662+0000 7f1547fff640 1 --2- 172.21.15.70:0/2001652876 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f154806b240 0x7f154808e4b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f153802a5f0 tx=0x7f15380046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:43:59.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.662+0000 7f1544ff9640 1 -- 172.21.15.70:0/2001652876 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538030850 con 0x7f154806b240 2024-09-13T15:43:59.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.662+0000 7f1544ff9640 1 -- 172.21.15.70:0/2001652876 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15380309b0 con 0x7f154806b240 2024-09-13T15:43:59.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.662+0000 7f154e533640 1 -- 172.21.15.70:0/2001652876 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1538009400 con 0x7f154806b240 2024-09-13T15:43:59.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.662+0000 7f1544ff9640 1 -- 172.21.15.70:0/2001652876 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538030d40 con 0x7f154806b240 2024-09-13T15:43:59.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.662+0000 7f154e533640 1 -- 172.21.15.70:0/2001652876 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1548090460 con 0x7f154806b240 2024-09-13T15:43:59.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.663+0000 7f1544ff9640 1 -- 172.21.15.70:0/2001652876 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f1538039040 con 0x7f154806b240 2024-09-13T15:43:59.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.663+0000 7f154e533640 1 -- 172.21.15.70:0/2001652876 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154806b620 con 0x7f154806b240 2024-09-13T15:43:59.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.664+0000 7f1544ff9640 1 -- 172.21.15.70:0/2001652876 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3023+0+0 (secure 0 0 0) 0x7f153807d4f0 con 0x7f154806b240 2024-09-13T15:43:59.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.667+0000 7f1544ff9640 1 -- 172.21.15.70:0/2001652876 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f1538047b70 con 0x7f154806b240 2024-09-13T15:43:59.813 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:43:59.810+0000 7f154e533640 1 -- 172.21.15.70:0/2001652876 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd require-osd-release", "release": "reef"} v 0) v1 -- 0x7f154808fd70 con 0x7f154806b240 2024-09-13T15:44:00.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.595+0000 7f1544ff9640 1 -- 172.21.15.70:0/2001652876 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd require-osd-release", "release": "reef"}]=0 v30) v1 ==== 90+0+0 (secure 0 0 0) 0x7f153805e710 con 0x7f154806b240 2024-09-13T15:44:00.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.597+0000 7f154e533640 1 -- 172.21.15.70:0/2001652876 >> v1:172.21.15.70:6816/1229510400 conn(0x7f1534047590 legacy=0x7f1534049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:00.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.597+0000 7f154e533640 1 -- 172.21.15.70:0/2001652876 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f154806b240 msgr2=0x7f154808e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:00.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.597+0000 7f154e533640 1 --2- 172.21.15.70:0/2001652876 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f154806b240 0x7f154808e4b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f153802a5f0 tx=0x7f15380046c0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.597+0000 7f154e533640 1 -- 172.21.15.70:0/2001652876 shutdown_connections 2024-09-13T15:44:00.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.597+0000 7f154e533640 1 --2- 172.21.15.70:0/2001652876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f154812b770 0x7f15481bff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.597+0000 7f154e533640 1 --2- 172.21.15.70:0/2001652876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f154806bb60 0x7f154808e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.597+0000 7f154e533640 1 --2- 172.21.15.70:0/2001652876 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f154806b240 0x7f154808e4b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.597+0000 7f154e533640 1 -- 172.21.15.70:0/2001652876 >> 172.21.15.70:0/2001652876 conn(0x7f1548098c40 msgr2=0x7f1548129cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:00.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.597+0000 7f154e533640 1 -- 172.21.15.70:0/2001652876 shutdown_connections 2024-09-13T15:44:00.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.597+0000 7f154e533640 1 -- 172.21.15.70:0/2001652876 wait complete. 2024-09-13T15:44:00.612 DEBUG:teuthology.orchestra.run.smithi070:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done' 2024-09-13T15:44:00.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.709+0000 7f29f275e640 1 Processor -- start 2024-09-13T15:44:00.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.709+0000 7f29f275e640 1 -- start start 2024-09-13T15:44:00.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.710+0000 7f29f275e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29ec129f40 0x7f29ec12a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:00.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.710+0000 7f29f275e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29ec06b950 0x7f29ec06bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:00.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.710+0000 7f29f275e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29ec06de60 con 0x7f29ec12a860 2024-09-13T15:44:00.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.710+0000 7f29f275e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ec06e230 con 0x7f29ec129f40 2024-09-13T15:44:00.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.710+0000 7f29f275e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec06cd40 con 0x7f29ec06b950 2024-09-13T15:44:00.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.710+0000 7f29ebfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29ec129f40 0x7f29ec12a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:00.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.710+0000 7f29f0cfb640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29ec06b950 0x7f29ec06bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:00.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.710+0000 7f29e3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f29ec12a860 0x7f29ec06b240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60436/0 (socket says 172.21.15.70:60436) 2024-09-13T15:44:00.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.710+0000 7f29e3fff640 1 -- 172.21.15.70:0/3792660346 learned_addr learned my addr 172.21.15.70:0/3792660346 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:00.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.710+0000 7f29ebfff640 1 -- 172.21.15.70:0/3792660346 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29ec06b950 msgr2=0x7f29ec06bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:00.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.710+0000 7f29ebfff640 1 --2- 172.21.15.70:0/3792660346 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29ec06b950 0x7f29ec06bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.710+0000 7f29ebfff640 1 -- 172.21.15.70:0/3792660346 >> v1:172.21.15.70:6789/0 conn(0x7f29ec12a860 legacy=0x7f29ec06b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:00.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.710+0000 7f29ebfff640 1 -- 172.21.15.70:0/3792660346 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec06d680 con 0x7f29ec129f40 2024-09-13T15:44:00.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.710+0000 7f29eb7fe640 1 -- 172.21.15.70:0/3792660346 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688642595 0 0) 0x7f29ec06de60 con 0x7f29ec12a860 2024-09-13T15:44:00.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.710+0000 7f29f0cfb640 1 --2- 172.21.15.70:0/3792660346 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29ec06b950 0x7f29ec06bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:00.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.711+0000 7f29ebfff640 1 --2- 172.21.15.70:0/3792660346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29ec129f40 0x7f29ec12a320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f29dc0097a0 tx=0x7f29dc02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f0bc0727d1ad05 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:00.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.711+0000 7f29eb7fe640 1 -- 172.21.15.70:0/3792660346 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc02adb0 con 0x7f29ec129f40 2024-09-13T15:44:00.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.711+0000 7f29eb7fe640 1 -- 172.21.15.70:0/3792660346 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc032440 con 0x7f29ec129f40 2024-09-13T15:44:00.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.711+0000 7f29eb7fe640 1 -- 172.21.15.70:0/3792660346 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0327d0 con 0x7f29ec129f40 2024-09-13T15:44:00.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.712+0000 7f29f275e640 1 -- 172.21.15.70:0/3792660346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29ec129f40 msgr2=0x7f29ec12a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:00.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.712+0000 7f29f275e640 1 --2- 172.21.15.70:0/3792660346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29ec129f40 0x7f29ec12a320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f29dc0097a0 tx=0x7f29dc02a060 comp rx=0 tx=0).stop 2024-09-13T15:44:00.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.712+0000 7f29f275e640 1 -- 172.21.15.70:0/3792660346 shutdown_connections 2024-09-13T15:44:00.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.712+0000 7f29f275e640 1 --2- 172.21.15.70:0/3792660346 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29ec06b950 0x7f29ec06bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.712+0000 7f29f275e640 1 --2- 172.21.15.70:0/3792660346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29ec129f40 0x7f29ec12a320 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.712+0000 7f29f275e640 1 -- 172.21.15.70:0/3792660346 >> 172.21.15.70:0/3792660346 conn(0x7f29ec098c40 msgr2=0x7f29ec096510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:00.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.712+0000 7f29f275e640 1 -- 172.21.15.70:0/3792660346 shutdown_connections 2024-09-13T15:44:00.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.712+0000 7f29f275e640 1 -- 172.21.15.70:0/3792660346 wait complete. 2024-09-13T15:44:00.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.712+0000 7f29f275e640 1 Processor -- start 2024-09-13T15:44:00.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29f275e640 1 -- start start 2024-09-13T15:44:00.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29f275e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29ec06b950 0x7f29ec1296d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:00.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29f275e640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29ec129f40 0x7f29ec120de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:00.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29f275e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29ec12a860 0x7f29ec121320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:00.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29f275e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec127930 con 0x7f29ec06b950 2024-09-13T15:44:00.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29f275e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec133a20 con 0x7f29ec12a860 2024-09-13T15:44:00.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29f275e640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec0708e0 con 0x7f29ec129f40 2024-09-13T15:44:00.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29ebfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29ec06b950 0x7f29ec1296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:00.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29e3fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29ec129f40 0x7f29ec120de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:00.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29ebfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29ec06b950 0x7f29ec1296d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40854/0 (socket says 172.21.15.70:40854) 2024-09-13T15:44:00.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29ebfff640 1 -- 172.21.15.70:0/2125135435 learned_addr learned my addr 172.21.15.70:0/2125135435 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:00.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29f0cfb640 1 --2- 172.21.15.70:0/2125135435 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29ec12a860 0x7f29ec121320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:00.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29ebfff640 1 -- 172.21.15.70:0/2125135435 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29ec129f40 msgr2=0x7f29ec120de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:00.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29ebfff640 1 --2- 172.21.15.70:0/2125135435 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29ec129f40 0x7f29ec120de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29ebfff640 1 -- 172.21.15.70:0/2125135435 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29ec12a860 msgr2=0x7f29ec121320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:00.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29ebfff640 1 --2- 172.21.15.70:0/2125135435 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29ec12a860 0x7f29ec121320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29ebfff640 1 -- 172.21.15.70:0/2125135435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e4009a70 con 0x7f29ec06b950 2024-09-13T15:44:00.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29e3fff640 1 --2- 172.21.15.70:0/2125135435 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29ec129f40 0x7f29ec120de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:00.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29f0cfb640 1 --2- 172.21.15.70:0/2125135435 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29ec12a860 0x7f29ec121320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:00.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.713+0000 7f29ebfff640 1 --2- 172.21.15.70:0/2125135435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29ec06b950 0x7f29ec1296d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f29dc02a580 tx=0x7f29dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:00.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.714+0000 7f29e97fa640 1 -- 172.21.15.70:0/2125135435 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc03d850 con 0x7f29ec06b950 2024-09-13T15:44:00.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.714+0000 7f29e97fa640 1 -- 172.21.15.70:0/2125135435 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc03d9d0 con 0x7f29ec06b950 2024-09-13T15:44:00.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.714+0000 7f29f275e640 1 -- 172.21.15.70:0/2125135435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29dc009400 con 0x7f29ec06b950 2024-09-13T15:44:00.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.714+0000 7f29e97fa640 1 -- 172.21.15.70:0/2125135435 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc03dd80 con 0x7f29ec06b950 2024-09-13T15:44:00.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.714+0000 7f29f275e640 1 -- 172.21.15.70:0/2125135435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec122aa0 con 0x7f29ec06b950 2024-09-13T15:44:00.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.715+0000 7f29e97fa640 1 -- 172.21.15.70:0/2125135435 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f29dc039040 con 0x7f29ec06b950 2024-09-13T15:44:00.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.715+0000 7f29f275e640 1 -- 172.21.15.70:0/2125135435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ec127480 con 0x7f29ec06b950 2024-09-13T15:44:00.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.715+0000 7f29e97fa640 1 -- 172.21.15.70:0/2125135435 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f29dc07d0b0 con 0x7f29ec06b950 2024-09-13T15:44:00.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.718+0000 7f29e97fa640 1 -- 172.21.15.70:0/2125135435 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f29ec127480 con 0x7f29ec06b950 2024-09-13T15:44:00.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.852+0000 7f29f275e640 1 -- 172.21.15.70:0/2125135435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool ls"} v 0) v1 -- 0x7f29ec127480 con 0x7f29ec06b950 2024-09-13T15:44:00.854 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.852+0000 7f29e97fa640 1 -- 172.21.15.70:0/2125135435 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v30) v1 ==== 59+0+9 (secure 0 0 0) 0x7f29ec127480 con 0x7f29ec06b950 2024-09-13T15:44:00.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.854+0000 7f29f275e640 1 -- 172.21.15.70:0/2125135435 >> v1:172.21.15.70:6816/1229510400 conn(0x7f29cc047560 legacy=0x7f29cc049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:00.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.854+0000 7f29f275e640 1 -- 172.21.15.70:0/2125135435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29ec06b950 msgr2=0x7f29ec1296d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:00.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.854+0000 7f29f275e640 1 --2- 172.21.15.70:0/2125135435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29ec06b950 0x7f29ec1296d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f29dc02a580 tx=0x7f29dc0046c0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.854+0000 7f29f275e640 1 -- 172.21.15.70:0/2125135435 shutdown_connections 2024-09-13T15:44:00.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.854+0000 7f29f275e640 1 --2- 172.21.15.70:0/2125135435 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29ec12a860 0x7f29ec121320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.854+0000 7f29f275e640 1 --2- 172.21.15.70:0/2125135435 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f29ec129f40 0x7f29ec120de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.854+0000 7f29f275e640 1 --2- 172.21.15.70:0/2125135435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29ec06b950 0x7f29ec1296d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.854+0000 7f29f275e640 1 -- 172.21.15.70:0/2125135435 >> 172.21.15.70:0/2125135435 conn(0x7f29ec098c40 msgr2=0x7f29ec09a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:00.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.854+0000 7f29f275e640 1 -- 172.21.15.70:0/2125135435 shutdown_connections 2024-09-13T15:44:00.857 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.854+0000 7f29f275e640 1 -- 172.21.15.70:0/2125135435 wait complete. 2024-09-13T15:44:00.944 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.936+0000 7f7f12480640 1 Processor -- start 2024-09-13T15:44:00.944 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.936+0000 7f7f12480640 1 -- start start 2024-09-13T15:44:00.944 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.936+0000 7f7f12480640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7f0c0955f0 0x7f7f0c0959d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:00.944 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.936+0000 7f7f12480640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f0c12a210 0x7f7f0c12a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.936+0000 7f7f12480640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7f0c12abb0 0x7f7f0c131540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.936+0000 7f7f12480640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f0c06e7d0 con 0x7f7f0c12abb0 2024-09-13T15:44:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.936+0000 7f7f12480640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c09b9f0 con 0x7f7f0c12a210 2024-09-13T15:44:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.936+0000 7f7f12480640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c06b7d0 con 0x7f7f0c0955f0 2024-09-13T15:44:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.937+0000 7f7f0bfff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7f0c0955f0 0x7f7f0c0959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.937+0000 7f7f0b7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f0c12a210 0x7f7f0c12a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.937+0000 7f7f10a1d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7f0c12abb0 0x7f7f0c131540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.937+0000 7f7f0b7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f0c12a210 0x7f7f0c12a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59394/0 (socket says 172.21.15.70:59394) 2024-09-13T15:44:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.937+0000 7f7f0bfff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7f0c0955f0 0x7f7f0c0959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58570/0 (socket says 172.21.15.70:58570) 2024-09-13T15:44:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.937+0000 7f7f0b7fe640 1 -- 172.21.15.70:0/827453125 learned_addr learned my addr 172.21.15.70:0/827453125 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.937+0000 7f7f10a1d640 1 -- 172.21.15.70:0/827453125 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7f0c0955f0 msgr2=0x7f7f0c0959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.937+0000 7f7f10a1d640 1 --2- 172.21.15.70:0/827453125 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7f0c0955f0 0x7f7f0c0959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.937+0000 7f7f10a1d640 1 -- 172.21.15.70:0/827453125 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f0c12a210 msgr2=0x7f7f0c12a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.937+0000 7f7f10a1d640 1 --2- 172.21.15.70:0/827453125 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f0c12a210 0x7f7f0c12a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.937+0000 7f7f10a1d640 1 -- 172.21.15.70:0/827453125 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c06e340 con 0x7f7f0c12abb0 2024-09-13T15:44:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.937+0000 7f7f0bfff640 1 --2- 172.21.15.70:0/827453125 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7f0c0955f0 0x7f7f0c0959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.937+0000 7f7f0b7fe640 1 --2- 172.21.15.70:0/827453125 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f0c12a210 0x7f7f0c12a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.938+0000 7f7f10a1d640 1 --2- 172.21.15.70:0/827453125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f0c12abb0 0x7f7f0c131540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009dc0 tx=0x7f7f0402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc203d08e0ca025 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.938+0000 7f7f0affd640 1 -- 172.21.15.70:0/827453125 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04007740 con 0x7f7f0c12abb0 2024-09-13T15:44:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.938+0000 7f7f0affd640 1 -- 172.21.15.70:0/827453125 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0402d050 con 0x7f7f0c12abb0 2024-09-13T15:44:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.938+0000 7f7f0affd640 1 -- 172.21.15.70:0/827453125 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04007ac0 con 0x7f7f0c12abb0 2024-09-13T15:44:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.938+0000 7f7f12480640 1 -- 172.21.15.70:0/827453125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f0c12abb0 msgr2=0x7f7f0c131540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.938+0000 7f7f12480640 1 --2- 172.21.15.70:0/827453125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f0c12abb0 0x7f7f0c131540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009dc0 tx=0x7f7f0402a560 comp rx=0 tx=0).stop 2024-09-13T15:44:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.938+0000 7f7f12480640 1 -- 172.21.15.70:0/827453125 shutdown_connections 2024-09-13T15:44:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.938+0000 7f7f12480640 1 --2- 172.21.15.70:0/827453125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f0c12abb0 0x7f7f0c131540 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.938+0000 7f7f12480640 1 --2- 172.21.15.70:0/827453125 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f0c12a210 0x7f7f0c12a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.938+0000 7f7f12480640 1 --2- 172.21.15.70:0/827453125 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7f0c0955f0 0x7f7f0c0959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.938+0000 7f7f12480640 1 -- 172.21.15.70:0/827453125 >> 172.21.15.70:0/827453125 conn(0x7f7f0c099a80 msgr2=0x7f7f0c099e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.938+0000 7f7f12480640 1 -- 172.21.15.70:0/827453125 shutdown_connections 2024-09-13T15:44:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.938+0000 7f7f12480640 1 -- 172.21.15.70:0/827453125 wait complete. 2024-09-13T15:44:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.939+0000 7f7f12480640 1 Processor -- start 2024-09-13T15:44:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.939+0000 7f7f12480640 1 -- start start 2024-09-13T15:44:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f12480640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f0c0955f0 0x7f7f0c0938c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f12480640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7f0c12a210 0x7f7f0c093e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:00.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f12480640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f0c12abb0 0x7f7f0c1c2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:00.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f12480640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c121f50 con 0x7f7f0c12abb0 2024-09-13T15:44:00.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f12480640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c094970 con 0x7f7f0c0955f0 2024-09-13T15:44:00.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f12480640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c095080 con 0x7f7f0c12a210 2024-09-13T15:44:00.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f0bfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f0c0955f0 0x7f7f0c0938c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:00.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f0b7fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7f0c12a210 0x7f7f0c093e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:00.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f0bfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f0c0955f0 0x7f7f0c0938c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59402/0 (socket says 172.21.15.70:59402) 2024-09-13T15:44:00.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f10a1d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f0c12abb0 0x7f7f0c1c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:00.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f0bfff640 1 -- 172.21.15.70:0/3400068285 learned_addr learned my addr 172.21.15.70:0/3400068285 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:00.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f0bfff640 1 -- 172.21.15.70:0/3400068285 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7f0c12a210 msgr2=0x7f7f0c093e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:00.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f0bfff640 1 --2- 172.21.15.70:0/3400068285 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7f0c12a210 0x7f7f0c093e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f0bfff640 1 -- 172.21.15.70:0/3400068285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f0c12abb0 msgr2=0x7f7f0c1c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:00.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f0bfff640 1 --2- 172.21.15.70:0/3400068285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f0c12abb0 0x7f7f0c1c2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:00.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f0bfff640 1 -- 172.21.15.70:0/3400068285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc009400 con 0x7f7f0c0955f0 2024-09-13T15:44:00.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f0bfff640 1 --2- 172.21.15.70:0/3400068285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f0c0955f0 0x7f7f0c0938c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00e980 tx=0x7f7efc00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:00.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f08ff9640 1 -- 172.21.15.70:0/3400068285 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0096e0 con 0x7f7f0c0955f0 2024-09-13T15:44:00.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f08ff9640 1 -- 172.21.15.70:0/3400068285 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc00ca00 con 0x7f7f0c0955f0 2024-09-13T15:44:00.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f12480640 1 -- 172.21.15.70:0/3400068285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f000094d0 con 0x7f7f0c0955f0 2024-09-13T15:44:00.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.940+0000 7f7f08ff9640 1 -- 172.21.15.70:0/3400068285 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc00cd90 con 0x7f7f0c0955f0 2024-09-13T15:44:00.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.941+0000 7f7f12480640 1 -- 172.21.15.70:0/3400068285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f04009a70 con 0x7f7f0c0955f0 2024-09-13T15:44:00.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.941+0000 7f7f08ff9640 1 -- 172.21.15.70:0/3400068285 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f7efc013470 con 0x7f7f0c0955f0 2024-09-13T15:44:00.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.942+0000 7f7f12480640 1 -- 172.21.15.70:0/3400068285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0c122350 con 0x7f7f0c0955f0 2024-09-13T15:44:00.951 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.942+0000 7f7f08ff9640 1 -- 172.21.15.70:0/3400068285 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f7efc05cb10 con 0x7f7f0c0955f0 2024-09-13T15:44:00.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:00.945+0000 7f7f08ff9640 1 -- 172.21.15.70:0/3400068285 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f7efc03a550 con 0x7f7f0c0955f0 2024-09-13T15:44:01.081 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.079+0000 7f7f12480640 1 -- 172.21.15.70:0/3400068285 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7f7f0c09a3e0 con 0x7f7f0c0955f0 2024-09-13T15:44:01.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.598+0000 7f7f08ff9640 1 -- 172.21.15.70:0/3400068285 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 1 pg_autoscale_mode to off v31) v1 ==== 153+0+0 (secure 0 0 0) 0x7f7efc03e380 con 0x7f7f0c0955f0 2024-09-13T15:44:01.601 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 1 pg_autoscale_mode to off 2024-09-13T15:44:01.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.600+0000 7f7f12480640 1 -- 172.21.15.70:0/3400068285 >> v1:172.21.15.70:6816/1229510400 conn(0x7f7ef8047590 legacy=0x7f7ef8049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:01.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.600+0000 7f7f12480640 1 -- 172.21.15.70:0/3400068285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f0c0955f0 msgr2=0x7f7f0c0938c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:01.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.600+0000 7f7f12480640 1 --2- 172.21.15.70:0/3400068285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f0c0955f0 0x7f7f0c0938c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00e980 tx=0x7f7efc00ee40 comp rx=0 tx=0).stop 2024-09-13T15:44:01.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.600+0000 7f7f12480640 1 -- 172.21.15.70:0/3400068285 shutdown_connections 2024-09-13T15:44:01.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.600+0000 7f7f12480640 1 --2- 172.21.15.70:0/3400068285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f0c12abb0 0x7f7f0c1c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:01.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.600+0000 7f7f12480640 1 --2- 172.21.15.70:0/3400068285 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f7f0c12a210 0x7f7f0c093e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:01.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.600+0000 7f7f12480640 1 --2- 172.21.15.70:0/3400068285 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f0c0955f0 0x7f7f0c0938c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:01.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.600+0000 7f7f12480640 1 -- 172.21.15.70:0/3400068285 >> 172.21.15.70:0/3400068285 conn(0x7f7f0c099a80 msgr2=0x7f7f0c096d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:01.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.600+0000 7f7f12480640 1 -- 172.21.15.70:0/3400068285 shutdown_connections 2024-09-13T15:44:01.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.600+0000 7f7f12480640 1 -- 172.21.15.70:0/3400068285 wait complete. 2024-09-13T15:44:01.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.686+0000 7ff87669c640 1 Processor -- start 2024-09-13T15:44:01.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.686+0000 7ff87669c640 1 -- start start 2024-09-13T15:44:01.689 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.687+0000 7ff87669c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff87006cae0 0x7ff87012f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:01.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.687+0000 7ff87669c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff87012f6a0 0x7ff870131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:01.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.687+0000 7ff87669c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87009b9f0 con 0x7ff87012a120 2024-09-13T15:44:01.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.687+0000 7ff87669c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff87006db30 con 0x7ff87012f6a0 2024-09-13T15:44:01.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.687+0000 7ff87669c640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff87006d1f0 con 0x7ff87006cae0 2024-09-13T15:44:01.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.687+0000 7ff86f7fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff87006cae0 0x7ff87012f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:01.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.687+0000 7ff86ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff87012a120 0x7ff87012a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60438/0 (socket says 172.21.15.70:60438) 2024-09-13T15:44:01.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.687+0000 7ff86ffff640 1 -- 172.21.15.70:0/2984342691 learned_addr learned my addr 172.21.15.70:0/2984342691 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:01.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.687+0000 7ff874c39640 1 --2- 172.21.15.70:0/2984342691 >> v2:172.21.15.70:3300/0 conn(0x7ff87012f6a0 0x7ff870131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:01.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.687+0000 7ff86f7fe640 1 -- 172.21.15.70:0/2984342691 >> v2:172.21.15.70:3300/0 conn(0x7ff87012f6a0 msgr2=0x7ff870131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:01.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.687+0000 7ff86f7fe640 1 --2- 172.21.15.70:0/2984342691 >> v2:172.21.15.70:3300/0 conn(0x7ff87012f6a0 0x7ff870131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:01.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.687+0000 7ff86f7fe640 1 -- 172.21.15.70:0/2984342691 >> v1:172.21.15.70:6789/0 conn(0x7ff87012a120 legacy=0x7ff87012a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:01.690 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.687+0000 7ff86f7fe640 1 -- 172.21.15.70:0/2984342691 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87006e320 con 0x7ff87006cae0 2024-09-13T15:44:01.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.688+0000 7ff86f7fe640 1 --2- 172.21.15.70:0/2984342691 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff87006cae0 0x7ff87012f160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8640097a0 tx=0x7ff86402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=926fa1ee74243c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:01.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.688+0000 7ff86effd640 1 -- 172.21.15.70:0/2984342691 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86402adb0 con 0x7ff87006cae0 2024-09-13T15:44:01.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.688+0000 7ff86effd640 1 -- 172.21.15.70:0/2984342691 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff864032440 con 0x7ff87006cae0 2024-09-13T15:44:01.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.688+0000 7ff86effd640 1 -- 172.21.15.70:0/2984342691 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8640327d0 con 0x7ff87006cae0 2024-09-13T15:44:01.693 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.688+0000 7ff87669c640 1 -- 172.21.15.70:0/2984342691 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff87006cae0 msgr2=0x7ff87012f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:01.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.688+0000 7ff87669c640 1 --2- 172.21.15.70:0/2984342691 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff87006cae0 0x7ff87012f160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8640097a0 tx=0x7ff86402a060 comp rx=0 tx=0).stop 2024-09-13T15:44:01.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.688+0000 7ff87669c640 1 -- 172.21.15.70:0/2984342691 shutdown_connections 2024-09-13T15:44:01.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.688+0000 7ff87669c640 1 --2- 172.21.15.70:0/2984342691 >> v2:172.21.15.70:3300/0 conn(0x7ff87012f6a0 0x7ff870131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:01.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.688+0000 7ff87669c640 1 --2- 172.21.15.70:0/2984342691 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff87006cae0 0x7ff87012f160 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:01.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.688+0000 7ff87669c640 1 -- 172.21.15.70:0/2984342691 >> 172.21.15.70:0/2984342691 conn(0x7ff870099a80 msgr2=0x7ff870099e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:01.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.688+0000 7ff87669c640 1 -- 172.21.15.70:0/2984342691 shutdown_connections 2024-09-13T15:44:01.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.688+0000 7ff87669c640 1 -- 172.21.15.70:0/2984342691 wait complete. 2024-09-13T15:44:01.694 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.689+0000 7ff87669c640 1 Processor -- start 2024-09-13T15:44:01.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.689+0000 7ff87669c640 1 -- start start 2024-09-13T15:44:01.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff87669c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff87006cae0 0x7ff870127440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:01.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff87669c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff87012a120 0x7ff870123150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:01.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff87669c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff87012f6a0 0x7ff870123690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:01.695 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff87669c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff870125610 con 0x7ff87012f6a0 2024-09-13T15:44:01.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff87669c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff870124c70 con 0x7ff87006cae0 2024-09-13T15:44:01.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff87669c640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8701268e0 con 0x7ff87012a120 2024-09-13T15:44:01.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff874c39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff87012f6a0 0x7ff870123690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:01.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff86f7fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff87012a120 0x7ff870123150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:01.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff86ffff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff87006cae0 0x7ff870127440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:01.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff874c39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff87012f6a0 0x7ff870123690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40890/0 (socket says 172.21.15.70:40890) 2024-09-13T15:44:01.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff874c39640 1 -- 172.21.15.70:0/562187490 learned_addr learned my addr 172.21.15.70:0/562187490 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:01.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff86f7fe640 1 -- 172.21.15.70:0/562187490 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff87006cae0 msgr2=0x7ff870127440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:01.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff86f7fe640 1 --2- 172.21.15.70:0/562187490 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff87006cae0 0x7ff870127440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:01.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff86f7fe640 1 -- 172.21.15.70:0/562187490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff87012f6a0 msgr2=0x7ff870123690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:01.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff86f7fe640 1 --2- 172.21.15.70:0/562187490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff87012f6a0 0x7ff870123690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:01.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff86f7fe640 1 -- 172.21.15.70:0/562187490 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff864009400 con 0x7ff87012a120 2024-09-13T15:44:01.696 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff86ffff640 1 --2- 172.21.15.70:0/562187490 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff87006cae0 0x7ff870127440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:01.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff874c39640 1 --2- 172.21.15.70:0/562187490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff87012f6a0 0x7ff870123690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:01.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.690+0000 7ff86f7fe640 1 --2- 172.21.15.70:0/562187490 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff87012a120 0x7ff870123150 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff864031000 tx=0x7ff8640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:01.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.691+0000 7ff86cff9640 1 -- 172.21.15.70:0/562187490 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86403d710 con 0x7ff87012a120 2024-09-13T15:44:01.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.691+0000 7ff86cff9640 1 -- 172.21.15.70:0/562187490 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86403d890 con 0x7ff87012a120 2024-09-13T15:44:01.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.691+0000 7ff87669c640 1 -- 172.21.15.70:0/562187490 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff870126b30 con 0x7ff87012a120 2024-09-13T15:44:01.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.691+0000 7ff86cff9640 1 -- 172.21.15.70:0/562187490 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86403dc40 con 0x7ff87012a120 2024-09-13T15:44:01.697 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.691+0000 7ff87669c640 1 -- 172.21.15.70:0/562187490 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff870127980 con 0x7ff87012a120 2024-09-13T15:44:01.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.692+0000 7ff86cff9640 1 -- 172.21.15.70:0/562187490 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7ff864039040 con 0x7ff87012a120 2024-09-13T15:44:01.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.692+0000 7ff87669c640 1 -- 172.21.15.70:0/562187490 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87012a500 con 0x7ff87012a120 2024-09-13T15:44:01.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.692+0000 7ff86cff9640 1 -- 172.21.15.70:0/562187490 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3027+0+0 (secure 0 0 0) 0x7ff86407d0a0 con 0x7ff87012a120 2024-09-13T15:44:01.698 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.695+0000 7ff86cff9640 1 -- 172.21.15.70:0/562187490 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7ff864034b90 con 0x7ff87012a120 2024-09-13T15:44:01.833 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:01.831+0000 7ff87669c640 1 -- 172.21.15.70:0/562187490 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7ff870131190 con 0x7ff87012a120 2024-09-13T15:44:02.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.600+0000 7ff86cff9640 1 -- 172.21.15.70:0/562187490 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 2 pg_autoscale_mode to off v32) v1 ==== 152+0+0 (secure 0 0 0) 0x7ff864041030 con 0x7ff87012a120 2024-09-13T15:44:02.602 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 2 pg_autoscale_mode to off 2024-09-13T15:44:02.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.603+0000 7ff87669c640 1 -- 172.21.15.70:0/562187490 >> v1:172.21.15.70:6816/1229510400 conn(0x7ff85c047560 legacy=0x7ff85c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:02.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.603+0000 7ff87669c640 1 -- 172.21.15.70:0/562187490 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff87012a120 msgr2=0x7ff870123150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:02.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.603+0000 7ff87669c640 1 --2- 172.21.15.70:0/562187490 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff87012a120 0x7ff870123150 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff864031000 tx=0x7ff8640046c0 comp rx=0 tx=0).stop 2024-09-13T15:44:02.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.603+0000 7ff87669c640 1 -- 172.21.15.70:0/562187490 shutdown_connections 2024-09-13T15:44:02.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.603+0000 7ff87669c640 1 --2- 172.21.15.70:0/562187490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff87012f6a0 0x7ff870123690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:02.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.603+0000 7ff87669c640 1 --2- 172.21.15.70:0/562187490 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff87012a120 0x7ff870123150 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:02.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.603+0000 7ff87669c640 1 --2- 172.21.15.70:0/562187490 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff87006cae0 0x7ff870127440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:02.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.603+0000 7ff87669c640 1 -- 172.21.15.70:0/562187490 >> 172.21.15.70:0/562187490 conn(0x7ff870099a80 msgr2=0x7ff870096da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:02.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.603+0000 7ff87669c640 1 -- 172.21.15.70:0/562187490 shutdown_connections 2024-09-13T15:44:02.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.603+0000 7ff87669c640 1 -- 172.21.15.70:0/562187490 wait complete. 2024-09-13T15:44:02.619 INFO:teuthology.run_tasks:Running task ceph.healthy... 2024-09-13T15:44:02.629 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T15:44:02.629 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T15:44:02.629 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T15:44:02.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.730+0000 7f9a202c5640 1 Processor -- start 2024-09-13T15:44:02.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.730+0000 7f9a202c5640 1 -- start start 2024-09-13T15:44:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a202c5640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a1806ddf0 0x7f9a1812f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a202c5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9a1812f960 0x7f9a18131d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a202c5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a18095ed0 con 0x7f9a180953e0 2024-09-13T15:44:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a202c5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a18097d40 con 0x7f9a1812f960 2024-09-13T15:44:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a202c5640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1806f110 con 0x7f9a1806ddf0 2024-09-13T15:44:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a1e862640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9a1812f960 0x7f9a18131d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a1d860640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a1806ddf0 0x7f9a1812f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a1e061640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9a180953e0 0x7f9a180957c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60444/0 (socket says 172.21.15.70:60444) 2024-09-13T15:44:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a1e862640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9a1812f960 0x7f9a18131d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40906/0 (socket says 172.21.15.70:40906) 2024-09-13T15:44:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a1e061640 1 -- 172.21.15.70:0/576676576 learned_addr learned my addr 172.21.15.70:0/576676576 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a1d860640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a1806ddf0 0x7f9a1812f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59420/0 (socket says 172.21.15.70:59420) 2024-09-13T15:44:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a1d860640 1 -- 172.21.15.70:0/576676576 >> v2:172.21.15.70:3300/0 conn(0x7f9a1812f960 msgr2=0x7f9a18131d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a1d860640 1 --2- 172.21.15.70:0/576676576 >> v2:172.21.15.70:3300/0 conn(0x7f9a1812f960 0x7f9a18131d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a1d860640 1 -- 172.21.15.70:0/576676576 >> v1:172.21.15.70:6789/0 conn(0x7f9a180953e0 legacy=0x7f9a180957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a1d860640 1 -- 172.21.15.70:0/576676576 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1806eb70 con 0x7f9a1806ddf0 2024-09-13T15:44:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a1d05f640 1 -- 172.21.15.70:0/576676576 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767511398 0 0) 0x7f9a18095ed0 con 0x7f9a180953e0 2024-09-13T15:44:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.731+0000 7f9a1e862640 1 --2- 172.21.15.70:0/576676576 >> v2:172.21.15.70:3300/0 conn(0x7f9a1812f960 0x7f9a18131d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.732+0000 7f9a1d860640 1 --2- 172.21.15.70:0/576676576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a1806ddf0 0x7f9a1812f420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9a080097a0 tx=0x7f9a0802a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0cf51a4d2b81d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.732+0000 7f9a1d05f640 1 -- 172.21.15.70:0/576676576 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0802adb0 con 0x7f9a1806ddf0 2024-09-13T15:44:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.732+0000 7f9a1d05f640 1 -- 172.21.15.70:0/576676576 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a08032440 con 0x7f9a1806ddf0 2024-09-13T15:44:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.732+0000 7f9a1d05f640 1 -- 172.21.15.70:0/576676576 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a080327d0 con 0x7f9a1806ddf0 2024-09-13T15:44:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.733+0000 7f9a202c5640 1 -- 172.21.15.70:0/576676576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a1806ddf0 msgr2=0x7f9a1812f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.733+0000 7f9a202c5640 1 --2- 172.21.15.70:0/576676576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a1806ddf0 0x7f9a1812f420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9a080097a0 tx=0x7f9a0802a060 comp rx=0 tx=0).stop 2024-09-13T15:44:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.733+0000 7f9a202c5640 1 -- 172.21.15.70:0/576676576 shutdown_connections 2024-09-13T15:44:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.733+0000 7f9a202c5640 1 --2- 172.21.15.70:0/576676576 >> v2:172.21.15.70:3300/0 conn(0x7f9a1812f960 0x7f9a18131d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.733+0000 7f9a202c5640 1 --2- 172.21.15.70:0/576676576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a1806ddf0 0x7f9a1812f420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.733+0000 7f9a202c5640 1 -- 172.21.15.70:0/576676576 >> 172.21.15.70:0/576676576 conn(0x7f9a18098b10 msgr2=0x7f9a180963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.733+0000 7f9a202c5640 1 -- 172.21.15.70:0/576676576 shutdown_connections 2024-09-13T15:44:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.733+0000 7f9a202c5640 1 -- 172.21.15.70:0/576676576 wait complete. 2024-09-13T15:44:02.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a202c5640 1 Processor -- start 2024-09-13T15:44:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a202c5640 1 -- start start 2024-09-13T15:44:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a202c5640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a1806ddf0 0x7f9a1806b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a202c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a180953e0 0x7f9a1806beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a202c5640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9a1812f960 0x7f9a1806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a202c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a18123290 con 0x7f9a180953e0 2024-09-13T15:44:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a202c5640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1812f040 con 0x7f9a1806ddf0 2024-09-13T15:44:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a202c5640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a18122d10 con 0x7f9a1812f960 2024-09-13T15:44:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a1e061640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a1806ddf0 0x7f9a1806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a1d860640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a180953e0 0x7f9a1806beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a1d860640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a180953e0 0x7f9a1806beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40908/0 (socket says 172.21.15.70:40908) 2024-09-13T15:44:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a1d860640 1 -- 172.21.15.70:0/3461727882 learned_addr learned my addr 172.21.15.70:0/3461727882 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a1e862640 1 --2- 172.21.15.70:0/3461727882 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9a1812f960 0x7f9a1806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a1d860640 1 -- 172.21.15.70:0/3461727882 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9a1812f960 msgr2=0x7f9a1806c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a1d860640 1 --2- 172.21.15.70:0/3461727882 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9a1812f960 0x7f9a1806c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a1d860640 1 -- 172.21.15.70:0/3461727882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a1806ddf0 msgr2=0x7f9a1806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a1d860640 1 --2- 172.21.15.70:0/3461727882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a1806ddf0 0x7f9a1806b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a1d860640 1 -- 172.21.15.70:0/3461727882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a10009a70 con 0x7f9a180953e0 2024-09-13T15:44:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.734+0000 7f9a1e061640 1 --2- 172.21.15.70:0/3461727882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a1806ddf0 0x7f9a1806b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.735+0000 7f9a1d860640 1 --2- 172.21.15.70:0/3461727882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a180953e0 0x7f9a1806beb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9a0802a580 tx=0x7f9a080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.735+0000 7f9a0effd640 1 -- 172.21.15.70:0/3461727882 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0803d850 con 0x7f9a180953e0 2024-09-13T15:44:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.735+0000 7f9a0effd640 1 -- 172.21.15.70:0/3461727882 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0803d9b0 con 0x7f9a180953e0 2024-09-13T15:44:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.735+0000 7f9a0effd640 1 -- 172.21.15.70:0/3461727882 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0803dd60 con 0x7f9a180953e0 2024-09-13T15:44:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.735+0000 7f9a202c5640 1 -- 172.21.15.70:0/3461727882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a08009400 con 0x7f9a180953e0 2024-09-13T15:44:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.735+0000 7f9a202c5640 1 -- 172.21.15.70:0/3461727882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1812a580 con 0x7f9a180953e0 2024-09-13T15:44:02.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.736+0000 7f9a202c5640 1 -- 172.21.15.70:0/3461727882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a180957c0 con 0x7f9a180953e0 2024-09-13T15:44:02.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.736+0000 7f9a0effd640 1 -- 172.21.15.70:0/3461727882 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f9a08039040 con 0x7f9a180953e0 2024-09-13T15:44:02.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.736+0000 7f9a0effd640 1 -- 172.21.15.70:0/3461727882 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f9a0807d0b0 con 0x7f9a180953e0 2024-09-13T15:44:02.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.740+0000 7f9a0effd640 1 -- 172.21.15.70:0/3461727882 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f9a08047d00 con 0x7f9a180953e0 2024-09-13T15:44:02.905 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.903+0000 7f9a202c5640 1 -- 172.21.15.70:0/3461727882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f9a181263d0 con 0x7f9a180953e0 2024-09-13T15:44:02.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.905+0000 7f9a0effd640 1 -- 172.21.15.70:0/3461727882 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v10) v1 ==== 74+0+107692 (secure 0 0 0) 0x7f9a08037090 con 0x7f9a180953e0 2024-09-13T15:44:02.909 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:44:02.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.907+0000 7f9a202c5640 1 -- 172.21.15.70:0/3461727882 >> v1:172.21.15.70:6816/1229510400 conn(0x7f99fc047560 legacy=0x7f99fc049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:02.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.907+0000 7f9a202c5640 1 -- 172.21.15.70:0/3461727882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a180953e0 msgr2=0x7f9a1806beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:02.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.907+0000 7f9a202c5640 1 --2- 172.21.15.70:0/3461727882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a180953e0 0x7f9a1806beb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9a0802a580 tx=0x7f9a080046c0 comp rx=0 tx=0).stop 2024-09-13T15:44:02.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.907+0000 7f9a202c5640 1 -- 172.21.15.70:0/3461727882 shutdown_connections 2024-09-13T15:44:02.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.907+0000 7f9a202c5640 1 --2- 172.21.15.70:0/3461727882 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9a1812f960 0x7f9a1806c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:02.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.907+0000 7f9a202c5640 1 --2- 172.21.15.70:0/3461727882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a180953e0 0x7f9a1806beb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:02.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.907+0000 7f9a202c5640 1 --2- 172.21.15.70:0/3461727882 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a1806ddf0 0x7f9a1806b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:02.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.907+0000 7f9a202c5640 1 -- 172.21.15.70:0/3461727882 >> 172.21.15.70:0/3461727882 conn(0x7f9a18098b10 msgr2=0x7f9a1806ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:02.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.907+0000 7f9a202c5640 1 -- 172.21.15.70:0/3461727882 shutdown_connections 2024-09-13T15:44:02.910 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:02.907+0000 7f9a202c5640 1 -- 172.21.15.70:0/3461727882 wait complete. 2024-09-13T15:44:02.920 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":10,"flags":0,"active_gid":24114,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6816","nonce":1229510400}]},"active_addr":"172.21.15.70:6816/1229510400","active_change":"2024-09-13T15:43:41.527139+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24105,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":19,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":1973565915}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":2316163223}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":2060411329}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":2689830311}]}]} 2024-09-13T15:44:02.922 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T15:44:02.923 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T15:44:02.923 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T15:44:03.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.017+0000 7faef5508640 1 Processor -- start 2024-09-13T15:44:03.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.017+0000 7faef5508640 1 -- start start 2024-09-13T15:44:03.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.017+0000 7faef5508640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faef006bc00 0x7faef006bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.017+0000 7faef5508640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7faef006c520 0x7faef012d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.017+0000 7faef5508640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef006c920 con 0x7faef012d740 2024-09-13T15:44:03.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.017+0000 7faef5508640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef0097940 con 0x7faef006bc00 2024-09-13T15:44:03.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.017+0000 7faef5508640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faef006d1f0 con 0x7faef006c520 2024-09-13T15:44:03.020 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.017+0000 7faee67fc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7faef006c520 0x7faef012d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.017+0000 7faeeeffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faef006bc00 0x7faef006bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.017+0000 7faeef7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7faef012d740 0x7faef012dbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60450/0 (socket says 172.21.15.70:60450) 2024-09-13T15:44:03.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.017+0000 7faeef7fe640 1 -- 172.21.15.70:0/4127044887 learned_addr learned my addr 172.21.15.70:0/4127044887 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:03.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.018+0000 7faee67fc640 1 -- 172.21.15.70:0/4127044887 >> v2:172.21.15.70:3300/0 conn(0x7faef006bc00 msgr2=0x7faef006bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.018+0000 7faee67fc640 1 --2- 172.21.15.70:0/4127044887 >> v2:172.21.15.70:3300/0 conn(0x7faef006bc00 0x7faef006bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.018+0000 7faee67fc640 1 -- 172.21.15.70:0/4127044887 >> v1:172.21.15.70:6789/0 conn(0x7faef012d740 legacy=0x7faef012dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.018+0000 7faee67fc640 1 -- 172.21.15.70:0/4127044887 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef006cf40 con 0x7faef006c520 2024-09-13T15:44:03.021 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.018+0000 7faeeeffd640 1 --2- 172.21.15.70:0/4127044887 >> v2:172.21.15.70:3300/0 conn(0x7faef006bc00 0x7faef006bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:03.024 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.018+0000 7faee67fc640 1 --2- 172.21.15.70:0/4127044887 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7faef006c520 0x7faef012d200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faed80097a0 tx=0x7faed802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce60b079a24493cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.018+0000 7faeee7fc640 1 -- 172.21.15.70:0/4127044887 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed802adb0 con 0x7faef006c520 2024-09-13T15:44:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.018+0000 7faeee7fc640 1 -- 172.21.15.70:0/4127044887 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed8032440 con 0x7faef006c520 2024-09-13T15:44:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.018+0000 7faeee7fc640 1 -- 172.21.15.70:0/4127044887 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed80327d0 con 0x7faef006c520 2024-09-13T15:44:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.019+0000 7faef5508640 1 -- 172.21.15.70:0/4127044887 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7faef006c520 msgr2=0x7faef012d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.019+0000 7faef5508640 1 --2- 172.21.15.70:0/4127044887 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7faef006c520 0x7faef012d200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faed80097a0 tx=0x7faed802a060 comp rx=0 tx=0).stop 2024-09-13T15:44:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.019+0000 7faef5508640 1 -- 172.21.15.70:0/4127044887 shutdown_connections 2024-09-13T15:44:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.019+0000 7faef5508640 1 --2- 172.21.15.70:0/4127044887 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7faef006c520 0x7faef012d200 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.019+0000 7faef5508640 1 --2- 172.21.15.70:0/4127044887 >> v2:172.21.15.70:3300/0 conn(0x7faef006bc00 0x7faef006bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.019+0000 7faef5508640 1 -- 172.21.15.70:0/4127044887 >> 172.21.15.70:0/4127044887 conn(0x7faef009fc40 msgr2=0x7faef00a0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.019+0000 7faef5508640 1 -- 172.21.15.70:0/4127044887 shutdown_connections 2024-09-13T15:44:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.019+0000 7faef5508640 1 -- 172.21.15.70:0/4127044887 wait complete. 2024-09-13T15:44:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.020+0000 7faef5508640 1 Processor -- start 2024-09-13T15:44:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.020+0000 7faef5508640 1 -- start start 2024-09-13T15:44:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.020+0000 7faef5508640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faef006bc00 0x7faef012eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.020+0000 7faef5508640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7faef006c520 0x7faef012f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.020+0000 7faef5508640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faef012d740 0x7faef012f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.020+0000 7faef5508640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef0130f70 con 0x7faef012d740 2024-09-13T15:44:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.020+0000 7faef5508640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef0099fb0 con 0x7faef006bc00 2024-09-13T15:44:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.020+0000 7faef5508640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faef0135230 con 0x7faef006c520 2024-09-13T15:44:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.020+0000 7faeeeffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faef006bc00 0x7faef012eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.020+0000 7faeef7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faef012d740 0x7faef012f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.020+0000 7faee67fc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7faef006c520 0x7faef012f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.021+0000 7faee67fc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7faef006c520 0x7faef012f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58612/0 (socket says 172.21.15.70:58612) 2024-09-13T15:44:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.021+0000 7faee67fc640 1 -- 172.21.15.70:0/3996188319 learned_addr learned my addr 172.21.15.70:0/3996188319 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.021+0000 7faeeeffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faef006bc00 0x7faef012eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59434/0 (socket says 172.21.15.70:59434) 2024-09-13T15:44:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.021+0000 7faeeeffd640 1 -- 172.21.15.70:0/3996188319 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7faef006c520 msgr2=0x7faef012f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.021+0000 7faeeeffd640 1 --2- 172.21.15.70:0/3996188319 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7faef006c520 0x7faef012f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.021+0000 7faeeeffd640 1 -- 172.21.15.70:0/3996188319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faef012d740 msgr2=0x7faef012f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.021+0000 7faeeeffd640 1 --2- 172.21.15.70:0/3996188319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faef012d740 0x7faef012f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.021+0000 7faeeeffd640 1 -- 172.21.15.70:0/3996188319 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee0009400 con 0x7faef006bc00 2024-09-13T15:44:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.021+0000 7faeef7fe640 1 --2- 172.21.15.70:0/3996188319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faef012d740 0x7faef012f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.021+0000 7faeeeffd640 1 --2- 172.21.15.70:0/3996188319 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faef006bc00 0x7faef012eb30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faee000e900 tx=0x7faee000edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.021+0000 7faee7fff640 1 -- 172.21.15.70:0/3996188319 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000ca00 con 0x7faef006bc00 2024-09-13T15:44:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.021+0000 7faee7fff640 1 -- 172.21.15.70:0/3996188319 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee000cb60 con 0x7faef006bc00 2024-09-13T15:44:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.021+0000 7faef5508640 1 -- 172.21.15.70:0/3996188319 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed8009400 con 0x7faef006bc00 2024-09-13T15:44:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.021+0000 7faee7fff640 1 -- 172.21.15.70:0/3996188319 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0013470 con 0x7faef006bc00 2024-09-13T15:44:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.021+0000 7faef5508640 1 -- 172.21.15.70:0/3996188319 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef0131890 con 0x7faef006bc00 2024-09-13T15:44:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.022+0000 7faee7fff640 1 -- 172.21.15.70:0/3996188319 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7faee00135d0 con 0x7faef006bc00 2024-09-13T15:44:03.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.022+0000 7faef5508640 1 -- 172.21.15.70:0/3996188319 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef006bfe0 con 0x7faef006bc00 2024-09-13T15:44:03.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.022+0000 7faee7fff640 1 -- 172.21.15.70:0/3996188319 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7faee005cb10 con 0x7faef006bc00 2024-09-13T15:44:03.028 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.026+0000 7faee7fff640 1 -- 172.21.15.70:0/3996188319 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7faee003a720 con 0x7faef006bc00 2024-09-13T15:44:03.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.161+0000 7faef5508640 1 -- 172.21.15.70:0/3996188319 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faef0127b30 con 0x7faef006bc00 2024-09-13T15:44:03.164 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.162+0000 7faee7fff640 1 -- 172.21.15.70:0/3996188319 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+9049 (secure 0 0 0) 0x7faee003dd40 con 0x7faef006bc00 2024-09-13T15:44:03.164 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:44:03.165 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":32,"fsid":"45bbcdad-f4fb-4639-ab42-ccda8de151dd","created":"2024-09-13T15:42:09.863982+0000","modified":"2024-09-13T15:44:02.597838+0000","last_up_change":"2024-09-13T15:43:55.573914+0000","last_in_change":"2024-09-13T15:42:11.714506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T15:42:21.000499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T15:42:26.149909+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"265c6f1d-08a7-4c43-b2a6-2417ab814825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":25,"up_thru":25,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":2246103037}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":2246103037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":2246103037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":2246103037}]},"public_addr":"172.21.15.70:6800/2246103037","cluster_addr":"172.21.15.70:6801/2246103037","heartbeat_back_addr":"172.21.15.70:6803/2246103037","heartbeat_front_addr":"172.21.15.70:6802/2246103037","state":["exists","up"]},{"osd":1,"uuid":"ba4d90f9-6972-45df-800f-92ad808b06ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":26,"up_thru":26,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":853418053}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":853418053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":853418053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":853418053}]},"public_addr":"172.21.15.70:6804/853418053","cluster_addr":"172.21.15.70:6805/853418053","heartbeat_back_addr":"172.21.15.70:6807/853418053","heartbeat_front_addr":"172.21.15.70:6806/853418053","state":["exists","up"]},{"osd":2,"uuid":"b74d8eb5-3082-4d69-a755-7f5f92dec5b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":27,"up_thru":28,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":2115999109}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":2115999109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":2115999109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":2115999109}]},"public_addr":"172.21.15.70:6808/2115999109","cluster_addr":"172.21.15.70:6809/2115999109","heartbeat_back_addr":"172.21.15.70:6811/2115999109","heartbeat_front_addr":"172.21.15.70:6810/2115999109","state":["exists","up"]},{"osd":3,"uuid":"02835b7d-e2cf-4ef8-87ba-b225593689d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":28,"up_thru":28,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":2095410362}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":2095410362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":2095410362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":2095410362}]},"public_addr":"172.21.15.70:6812/2095410362","cluster_addr":"172.21.15.70:6813/2095410362","heartbeat_back_addr":"172.21.15.70:6815/2095410362","heartbeat_front_addr":"172.21.15.70:6814/2095410362","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.853420+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.807491+0000","dead_epoch":21},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.925567+0000","dead_epoch":22},{"osd":3,"down_stamp":"2024-09-13T15:43:48.549448+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.883852+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6816/32242":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/2914106548":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3671770748":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3345804336":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3873928986":"2024-09-14T15:43:41.526957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T15:44:03.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.164+0000 7faef5508640 1 -- 172.21.15.70:0/3996188319 >> v1:172.21.15.70:6816/1229510400 conn(0x7faed0047560 legacy=0x7faed0049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.164+0000 7faef5508640 1 -- 172.21.15.70:0/3996188319 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faef006bc00 msgr2=0x7faef012eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.164+0000 7faef5508640 1 --2- 172.21.15.70:0/3996188319 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faef006bc00 0x7faef012eb30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faee000e900 tx=0x7faee000edc0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.164+0000 7faef5508640 1 -- 172.21.15.70:0/3996188319 shutdown_connections 2024-09-13T15:44:03.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.164+0000 7faef5508640 1 --2- 172.21.15.70:0/3996188319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faef012d740 0x7faef012f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.166 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.164+0000 7faef5508640 1 --2- 172.21.15.70:0/3996188319 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7faef006c520 0x7faef012f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.164+0000 7faef5508640 1 --2- 172.21.15.70:0/3996188319 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faef006bc00 0x7faef012eb30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.164+0000 7faef5508640 1 -- 172.21.15.70:0/3996188319 >> 172.21.15.70:0/3996188319 conn(0x7faef009fc40 msgr2=0x7faef0124d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.164+0000 7faef5508640 1 -- 172.21.15.70:0/3996188319 shutdown_connections 2024-09-13T15:44:03.167 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.164+0000 7faef5508640 1 -- 172.21.15.70:0/3996188319 wait complete. 2024-09-13T15:44:03.177 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T15:44:03.177 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T15:44:03.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.271+0000 7fb9639bc640 1 Processor -- start 2024-09-13T15:44:03.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.271+0000 7fb9639bc640 1 -- start start 2024-09-13T15:44:03.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.272+0000 7fb9639bc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb95c06b970 0x7fb95c06bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.272+0000 7fb9639bc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb95c06c290 0x7fb95c095090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.272+0000 7fb9639bc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95c065e10 con 0x7fb95c0955d0 2024-09-13T15:44:03.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.272+0000 7fb9639bc640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95c097d40 con 0x7fb95c06c290 2024-09-13T15:44:03.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.272+0000 7fb9639bc640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c0702c0 con 0x7fb95c06b970 2024-09-13T15:44:03.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.272+0000 7fb961758640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb95c06b970 0x7fb95c06bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.272+0000 7fb960f57640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb95c06c290 0x7fb95c095090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.272+0000 7fb961f59640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb95c0955d0 0x7fb95c132740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60452/0 (socket says 172.21.15.70:60452) 2024-09-13T15:44:03.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.272+0000 7fb961f59640 1 -- 172.21.15.70:0/3783605173 learned_addr learned my addr 172.21.15.70:0/3783605173 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:03.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.272+0000 7fb961758640 1 -- 172.21.15.70:0/3783605173 >> v2:172.21.15.70:3300/0 conn(0x7fb95c06c290 msgr2=0x7fb95c095090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.272+0000 7fb961758640 1 --2- 172.21.15.70:0/3783605173 >> v2:172.21.15.70:3300/0 conn(0x7fb95c06c290 0x7fb95c095090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.272+0000 7fb961758640 1 -- 172.21.15.70:0/3783605173 >> v1:172.21.15.70:6789/0 conn(0x7fb95c0955d0 legacy=0x7fb95c132740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.272+0000 7fb961758640 1 -- 172.21.15.70:0/3783605173 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c06c6d0 con 0x7fb95c06b970 2024-09-13T15:44:03.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.272+0000 7fb960f57640 1 --2- 172.21.15.70:0/3783605173 >> v2:172.21.15.70:3300/0 conn(0x7fb95c06c290 0x7fb95c095090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:03.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.273+0000 7fb961758640 1 --2- 172.21.15.70:0/3783605173 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb95c06b970 0x7fb95c06bd50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb9580097a0 tx=0x7fb95802a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=842800c02bf4ca9a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.273+0000 7fb953fff640 1 -- 172.21.15.70:0/3783605173 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95802adb0 con 0x7fb95c06b970 2024-09-13T15:44:03.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.273+0000 7fb953fff640 1 -- 172.21.15.70:0/3783605173 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb958032440 con 0x7fb95c06b970 2024-09-13T15:44:03.277 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.273+0000 7fb953fff640 1 -- 172.21.15.70:0/3783605173 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9580327d0 con 0x7fb95c06b970 2024-09-13T15:44:03.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.273+0000 7fb9639bc640 1 -- 172.21.15.70:0/3783605173 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb95c06b970 msgr2=0x7fb95c06bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.273+0000 7fb9639bc640 1 --2- 172.21.15.70:0/3783605173 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb95c06b970 0x7fb95c06bd50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb9580097a0 tx=0x7fb95802a060 comp rx=0 tx=0).stop 2024-09-13T15:44:03.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.273+0000 7fb9639bc640 1 -- 172.21.15.70:0/3783605173 shutdown_connections 2024-09-13T15:44:03.280 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.273+0000 7fb9639bc640 1 --2- 172.21.15.70:0/3783605173 >> v2:172.21.15.70:3300/0 conn(0x7fb95c06c290 0x7fb95c095090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.273+0000 7fb9639bc640 1 --2- 172.21.15.70:0/3783605173 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb95c06b970 0x7fb95c06bd50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.273+0000 7fb9639bc640 1 -- 172.21.15.70:0/3783605173 >> 172.21.15.70:0/3783605173 conn(0x7fb95c098b10 msgr2=0x7fb95c0963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.274+0000 7fb9639bc640 1 -- 172.21.15.70:0/3783605173 shutdown_connections 2024-09-13T15:44:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.274+0000 7fb9639bc640 1 -- 172.21.15.70:0/3783605173 wait complete. 2024-09-13T15:44:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.274+0000 7fb9639bc640 1 Processor -- start 2024-09-13T15:44:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.274+0000 7fb9639bc640 1 -- start start 2024-09-13T15:44:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb9639bc640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb95c06b970 0x7fb95c1c2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb9639bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb95c06c290 0x7fb95c1c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb9639bc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb95c0955d0 0x7fb95c1c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb9639bc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c08f6c0 con 0x7fb95c06c290 2024-09-13T15:44:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb9639bc640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c1282a0 con 0x7fb95c0955d0 2024-09-13T15:44:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb9639bc640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c06e4c0 con 0x7fb95c06b970 2024-09-13T15:44:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb961758640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb95c06b970 0x7fb95c1c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb960f57640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb95c06c290 0x7fb95c1c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb961f59640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb95c0955d0 0x7fb95c1c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb961758640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb95c06b970 0x7fb95c1c2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58614/0 (socket says 172.21.15.70:58614) 2024-09-13T15:44:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb961758640 1 -- 172.21.15.70:0/1043878992 learned_addr learned my addr 172.21.15.70:0/1043878992 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb960f57640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb95c06c290 0x7fb95c1c2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40926/0 (socket says 172.21.15.70:40926) 2024-09-13T15:44:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb961758640 1 -- 172.21.15.70:0/1043878992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb95c0955d0 msgr2=0x7fb95c1c2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb961758640 1 --2- 172.21.15.70:0/1043878992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb95c0955d0 0x7fb95c1c2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb961758640 1 -- 172.21.15.70:0/1043878992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb95c06c290 msgr2=0x7fb95c1c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb961758640 1 --2- 172.21.15.70:0/1043878992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb95c06c290 0x7fb95c1c2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb961758640 1 -- 172.21.15.70:0/1043878992 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c009400 con 0x7fb95c06b970 2024-09-13T15:44:03.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb961f59640 1 --2- 172.21.15.70:0/1043878992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb95c0955d0 0x7fb95c1c2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:03.283 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb961758640 1 --2- 172.21.15.70:0/1043878992 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb95c06b970 0x7fb95c1c2530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb95802a580 tx=0x7fb9580046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.275+0000 7fb951ffb640 1 -- 172.21.15.70:0/1043878992 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95803d850 con 0x7fb95c06b970 2024-09-13T15:44:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.276+0000 7fb951ffb640 1 -- 172.21.15.70:0/1043878992 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95803d9b0 con 0x7fb95c06b970 2024-09-13T15:44:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.276+0000 7fb9639bc640 1 -- 172.21.15.70:0/1043878992 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb958009400 con 0x7fb95c06b970 2024-09-13T15:44:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.276+0000 7fb951ffb640 1 -- 172.21.15.70:0/1043878992 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95803dd60 con 0x7fb95c06b970 2024-09-13T15:44:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.276+0000 7fb9639bc640 1 -- 172.21.15.70:0/1043878992 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95c125430 con 0x7fb95c06b970 2024-09-13T15:44:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.277+0000 7fb951ffb640 1 -- 172.21.15.70:0/1043878992 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fb958039040 con 0x7fb95c06b970 2024-09-13T15:44:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.277+0000 7fb9639bc640 1 -- 172.21.15.70:0/1043878992 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95c095dd0 con 0x7fb95c06b970 2024-09-13T15:44:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.277+0000 7fb951ffb640 1 -- 172.21.15.70:0/1043878992 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fb95807d1d0 con 0x7fb95c06b970 2024-09-13T15:44:03.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.281+0000 7fb951ffb640 1 -- 172.21.15.70:0/1043878992 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fb958047b30 con 0x7fb95c06b970 2024-09-13T15:44:03.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.417+0000 7fb9639bc640 1 -- 172.21.15.70:0/1043878992 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb95c066fa0 con 0x7fb95c06b970 2024-09-13T15:44:03.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.417+0000 7fb951ffb640 1 -- 172.21.15.70:0/1043878992 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+9049 (secure 0 0 0) 0x7fb958041090 con 0x7fb95c06b970 2024-09-13T15:44:03.420 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:44:03.420 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":32,"fsid":"45bbcdad-f4fb-4639-ab42-ccda8de151dd","created":"2024-09-13T15:42:09.863982+0000","modified":"2024-09-13T15:44:02.597838+0000","last_up_change":"2024-09-13T15:43:55.573914+0000","last_in_change":"2024-09-13T15:42:11.714506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T15:42:21.000499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T15:42:26.149909+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"265c6f1d-08a7-4c43-b2a6-2417ab814825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":25,"up_thru":25,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":2246103037}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":2246103037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":2246103037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":2246103037}]},"public_addr":"172.21.15.70:6800/2246103037","cluster_addr":"172.21.15.70:6801/2246103037","heartbeat_back_addr":"172.21.15.70:6803/2246103037","heartbeat_front_addr":"172.21.15.70:6802/2246103037","state":["exists","up"]},{"osd":1,"uuid":"ba4d90f9-6972-45df-800f-92ad808b06ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":26,"up_thru":26,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":853418053}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":853418053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":853418053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":853418053}]},"public_addr":"172.21.15.70:6804/853418053","cluster_addr":"172.21.15.70:6805/853418053","heartbeat_back_addr":"172.21.15.70:6807/853418053","heartbeat_front_addr":"172.21.15.70:6806/853418053","state":["exists","up"]},{"osd":2,"uuid":"b74d8eb5-3082-4d69-a755-7f5f92dec5b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":27,"up_thru":28,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":2115999109}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":2115999109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":2115999109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":2115999109}]},"public_addr":"172.21.15.70:6808/2115999109","cluster_addr":"172.21.15.70:6809/2115999109","heartbeat_back_addr":"172.21.15.70:6811/2115999109","heartbeat_front_addr":"172.21.15.70:6810/2115999109","state":["exists","up"]},{"osd":3,"uuid":"02835b7d-e2cf-4ef8-87ba-b225593689d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":28,"up_thru":28,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":2095410362}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":2095410362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":2095410362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":2095410362}]},"public_addr":"172.21.15.70:6812/2095410362","cluster_addr":"172.21.15.70:6813/2095410362","heartbeat_back_addr":"172.21.15.70:6815/2095410362","heartbeat_front_addr":"172.21.15.70:6814/2095410362","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.853420+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.807491+0000","dead_epoch":21},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.925567+0000","dead_epoch":22},{"osd":3,"down_stamp":"2024-09-13T15:43:48.549448+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.883852+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6816/32242":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/2914106548":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3671770748":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3345804336":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3873928986":"2024-09-14T15:43:41.526957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T15:44:03.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.419+0000 7fb9639bc640 1 -- 172.21.15.70:0/1043878992 >> v1:172.21.15.70:6816/1229510400 conn(0x7fb940047560 legacy=0x7fb940049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.419+0000 7fb9639bc640 1 -- 172.21.15.70:0/1043878992 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb95c06b970 msgr2=0x7fb95c1c2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.419+0000 7fb9639bc640 1 --2- 172.21.15.70:0/1043878992 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb95c06b970 0x7fb95c1c2530 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb95802a580 tx=0x7fb9580046c0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.419+0000 7fb9639bc640 1 -- 172.21.15.70:0/1043878992 shutdown_connections 2024-09-13T15:44:03.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.419+0000 7fb9639bc640 1 --2- 172.21.15.70:0/1043878992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb95c0955d0 0x7fb95c1c2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.419+0000 7fb9639bc640 1 --2- 172.21.15.70:0/1043878992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb95c06c290 0x7fb95c1c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.419+0000 7fb9639bc640 1 --2- 172.21.15.70:0/1043878992 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb95c06b970 0x7fb95c1c2530 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.419+0000 7fb9639bc640 1 -- 172.21.15.70:0/1043878992 >> 172.21.15.70:0/1043878992 conn(0x7fb95c098b10 msgr2=0x7fb95c099860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.419+0000 7fb9639bc640 1 -- 172.21.15.70:0/1043878992 shutdown_connections 2024-09-13T15:44:03.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.419+0000 7fb9639bc640 1 -- 172.21.15.70:0/1043878992 wait complete. 2024-09-13T15:44:03.433 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T15:44:03.434 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T15:44:03.435 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T15:44:03.435 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T15:44:03.539 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.537+0000 7fe8934d0640 1 Processor -- start 2024-09-13T15:44:03.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.537+0000 7fbe1823a640 1 Processor -- start 2024-09-13T15:44:03.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.537+0000 7f56151ed640 1 Processor -- start 2024-09-13T15:44:03.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.537+0000 7f56151ed640 1 -- start start 2024-09-13T15:44:03.540 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.537+0000 7f889a757640 1 Processor -- start 2024-09-13T15:44:03.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fe8934d0640 1 -- start start 2024-09-13T15:44:03.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7f889a757640 1 -- start start 2024-09-13T15:44:03.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fbe1823a640 1 -- start start 2024-09-13T15:44:03.542 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7f56151ed640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f561012d4d0 0x7f561012d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.542 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7f56151ed640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f561012ddf0 0x7f5610129fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.542 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fe8934d0640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe88c06b970 0x7fe88c06bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.542 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fe8934d0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe88c0957a0 0x7fe88c133740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.542 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fe8934d0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88c06fd20 con 0x7fe88c06c290 2024-09-13T15:44:03.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fe8934d0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe88c097d40 con 0x7fe88c0957a0 2024-09-13T15:44:03.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fe8934d0640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c070270 con 0x7fe88c06b970 2024-09-13T15:44:03.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7f56151ed640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f561012a510 0x7f561012a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7f56151ed640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56100977c0 con 0x7f561012d4d0 2024-09-13T15:44:03.543 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7f56151ed640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561012aee0 con 0x7f561012ddf0 2024-09-13T15:44:03.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7f889a757640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889406ca80 0x7f889406ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fbe1823a640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbe1012d330 0x7fbe1012d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.544 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7f889a757640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f889406d3a0 0x7f88940957a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7f889a757640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8894097d40 con 0x7f889406dd10 2024-09-13T15:44:03.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7f889a757640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889406f1d0 con 0x7f889406ca80 2024-09-13T15:44:03.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7f889a757640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f889406ebc0 con 0x7f889406d3a0 2024-09-13T15:44:03.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fe89126c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe88c06b970 0x7fe88c06bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fe89126c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe88c06b970 0x7fe88c06bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58640/0 (socket says 172.21.15.70:58640) 2024-09-13T15:44:03.545 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7fe89126c640 1 -- 172.21.15.70:0/704283609 learned_addr learned my addr 172.21.15.70:0/704283609 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7f56151ed640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56100971b0 con 0x7f561012a510 2024-09-13T15:44:03.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7f560ed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f561012d4d0 0x7f561012d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7f560ed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f561012d4d0 0x7f561012d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40938/0 (socket says 172.21.15.70:40938) 2024-09-13T15:44:03.546 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7f560ed9d640 1 -- 172.21.15.70:0/585704942 learned_addr learned my addr 172.21.15.70:0/585704942 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f8893fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889406ca80 0x7f889406ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.547 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f560e59c640 1 --2- 172.21.15.70:0/585704942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f561012ddf0 0x7f5610129fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f560f59e640 1 --2- 172.21.15.70:0/585704942 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f561012a510 0x7f561012a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f8893fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889406ca80 0x7f889406ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59462/0 (socket says 172.21.15.70:59462) 2024-09-13T15:44:03.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7fe891a6d640 1 --2- 172.21.15.70:0/704283609 >> v2:172.21.15.70:3300/0 conn(0x7fe88c0957a0 0x7fe88c133740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f8893fff640 1 -- 172.21.15.70:0/2982608379 learned_addr learned my addr 172.21.15.70:0/2982608379 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f8898cf4640 1 --2- 172.21.15.70:0/2982608379 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f889406d3a0 0x7f88940957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.549 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f8893fff640 1 -- 172.21.15.70:0/2982608379 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f889406d3a0 msgr2=0x7f88940957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f560ed9d640 1 -- 172.21.15.70:0/585704942 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f561012a510 msgr2=0x7f561012a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f560ed9d640 1 --2- 172.21.15.70:0/585704942 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f561012a510 0x7f561012a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f560ed9d640 1 -- 172.21.15.70:0/585704942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f561012ddf0 msgr2=0x7f5610129fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.550 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f560ed9d640 1 --2- 172.21.15.70:0/585704942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f561012ddf0 0x7f5610129fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f560ed9d640 1 -- 172.21.15.70:0/585704942 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561009a100 con 0x7f561012d4d0 2024-09-13T15:44:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f8893fff640 1 --2- 172.21.15.70:0/2982608379 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f889406d3a0 0x7f88940957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f8893fff640 1 -- 172.21.15.70:0/2982608379 >> v1:172.21.15.70:6789/0 conn(0x7f889406dd10 legacy=0x7f8894095090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f8893fff640 1 -- 172.21.15.70:0/2982608379 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889406f400 con 0x7f889406ca80 2024-09-13T15:44:03.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7f8898cf4640 1 --2- 172.21.15.70:0/2982608379 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f889406d3a0 0x7f88940957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:44:03.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fbe1823a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbe1012a540 0x7fbe10135670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fbe1823a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe1006d5d0 con 0x7fbe1012dd80 2024-09-13T15:44:03.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fbe1823a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe100971b0 con 0x7fbe1012a540 2024-09-13T15:44:03.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fbe1823a640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe100979f0 con 0x7fbe1012d330 2024-09-13T15:44:03.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fbe15fd6640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbe1012d330 0x7fbe1012d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fbe15fd6640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbe1012d330 0x7fbe1012d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58652/0 (socket says 172.21.15.70:58652) 2024-09-13T15:44:03.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.538+0000 7fbe15fd6640 1 -- 172.21.15.70:0/4068875997 learned_addr learned my addr 172.21.15.70:0/4068875997 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7fbe167d7640 1 --2- 172.21.15.70:0/4068875997 >> v2:172.21.15.70:3300/0 conn(0x7fbe1012a540 0x7fbe10135670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7fbe167d7640 1 -- 172.21.15.70:0/4068875997 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbe1012d330 msgr2=0x7fbe1012d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7fbe167d7640 1 --2- 172.21.15.70:0/4068875997 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbe1012d330 0x7fbe1012d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7fbe167d7640 1 -- 172.21.15.70:0/4068875997 >> v1:172.21.15.70:6789/0 conn(0x7fbe1012dd80 legacy=0x7fbe10129e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.539+0000 7fbe167d7640 1 -- 172.21.15.70:0/4068875997 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1012e1a0 con 0x7fbe1012a540 2024-09-13T15:44:03.553 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7fe89126c640 1 -- 172.21.15.70:0/704283609 >> v2:172.21.15.70:3300/0 conn(0x7fe88c0957a0 msgr2=0x7fe88c133740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7fe89126c640 1 --2- 172.21.15.70:0/704283609 >> v2:172.21.15.70:3300/0 conn(0x7fe88c0957a0 0x7fe88c133740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7fe89126c640 1 -- 172.21.15.70:0/704283609 >> v1:172.21.15.70:6789/0 conn(0x7fe88c06c290 legacy=0x7fe88c095090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7fe883fff640 1 -- 172.21.15.70:0/704283609 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986427708 0 0) 0x7fe88c06fd20 con 0x7fe88c06c290 2024-09-13T15:44:03.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7fe89126c640 1 -- 172.21.15.70:0/704283609 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c098130 con 0x7fe88c06b970 2024-09-13T15:44:03.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7fbe167d7640 1 --2- 172.21.15.70:0/4068875997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe1012a540 0x7fbe10135670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009d90 tx=0x7fbe0802a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beaaaf9c7536a981 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7fbe14fd4640 1 -- 172.21.15.70:0/4068875997 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08007740 con 0x7fbe1012a540 2024-09-13T15:44:03.554 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7fbe14fd4640 1 -- 172.21.15.70:0/4068875997 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0802d050 con 0x7fbe1012a540 2024-09-13T15:44:03.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7f560ed9d640 1 --2- 172.21.15.70:0/585704942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f561012d4d0 0x7f561012d8b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5600009870 tx=0x7f560002a090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7b5125162c66ae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7fbe14fd4640 1 -- 172.21.15.70:0/4068875997 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08007ac0 con 0x7fbe1012a540 2024-09-13T15:44:03.555 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7f8893fff640 1 --2- 172.21.15.70:0/2982608379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889406ca80 0x7f889406ce60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f88840098a0 tx=0x7f888402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7877c1a09ab4ddb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7f560dd9b640 1 -- 172.21.15.70:0/585704942 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560002ade0 con 0x7f561012d4d0 2024-09-13T15:44:03.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7f8892ffd640 1 -- 172.21.15.70:0/2982608379 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888402ae60 con 0x7f889406ca80 2024-09-13T15:44:03.556 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7f560dd9b640 1 -- 172.21.15.70:0/585704942 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5600032440 con 0x7f561012d4d0 2024-09-13T15:44:03.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7f8892ffd640 1 -- 172.21.15.70:0/2982608379 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8884032440 con 0x7f889406ca80 2024-09-13T15:44:03.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.540+0000 7f8892ffd640 1 -- 172.21.15.70:0/2982608379 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88840327d0 con 0x7f889406ca80 2024-09-13T15:44:03.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f560dd9b640 1 -- 172.21.15.70:0/585704942 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560002ade0 con 0x7f561012d4d0 2024-09-13T15:44:03.557 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7fe89126c640 1 --2- 172.21.15.70:0/704283609 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe88c06b970 0x7fe88c06bd50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe888009870 tx=0x7fe88802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf5abe6f5fb34b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7fbe1823a640 1 -- 172.21.15.70:0/4068875997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe1012a540 msgr2=0x7fbe10135670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7fbe1823a640 1 --2- 172.21.15.70:0/4068875997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe1012a540 0x7fbe10135670 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009d90 tx=0x7fbe0802a4e0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7fbe1823a640 1 -- 172.21.15.70:0/4068875997 shutdown_connections 2024-09-13T15:44:03.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7fbe1823a640 1 --2- 172.21.15.70:0/4068875997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe1012a540 0x7fbe10135670 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7fbe1823a640 1 --2- 172.21.15.70:0/4068875997 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbe1012d330 0x7fbe1012d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.558 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7fbe1823a640 1 -- 172.21.15.70:0/4068875997 >> 172.21.15.70:0/4068875997 conn(0x7fbe100a0060 msgr2=0x7fbe100a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7fe883fff640 1 -- 172.21.15.70:0/704283609 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888032440 con 0x7fe88c06b970 2024-09-13T15:44:03.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7fe883fff640 1 -- 172.21.15.70:0/704283609 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8880325a0 con 0x7fe88c06b970 2024-09-13T15:44:03.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7fe883fff640 1 -- 172.21.15.70:0/704283609 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888032930 con 0x7fe88c06b970 2024-09-13T15:44:03.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7fbe1823a640 1 -- 172.21.15.70:0/4068875997 shutdown_connections 2024-09-13T15:44:03.560 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f56151ed640 1 -- 172.21.15.70:0/585704942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f561012d4d0 msgr2=0x7f561012d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.560 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f56151ed640 1 --2- 172.21.15.70:0/585704942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f561012d4d0 0x7f561012d8b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5600009870 tx=0x7f560002a090 comp rx=0 tx=0).stop 2024-09-13T15:44:03.560 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f889a757640 1 -- 172.21.15.70:0/2982608379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889406ca80 msgr2=0x7f889406ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.560 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f889a757640 1 --2- 172.21.15.70:0/2982608379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889406ca80 0x7f889406ce60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f88840098a0 tx=0x7f888402a110 comp rx=0 tx=0).stop 2024-09-13T15:44:03.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f56151ed640 1 -- 172.21.15.70:0/585704942 shutdown_connections 2024-09-13T15:44:03.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f56151ed640 1 --2- 172.21.15.70:0/585704942 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f561012a510 0x7f561012a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f56151ed640 1 --2- 172.21.15.70:0/585704942 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f561012ddf0 0x7f5610129fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f56151ed640 1 --2- 172.21.15.70:0/585704942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f561012d4d0 0x7f561012d8b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f56151ed640 1 -- 172.21.15.70:0/585704942 >> 172.21.15.70:0/585704942 conn(0x7f56100a0060 msgr2=0x7f56100a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7fbe1823a640 1 -- 172.21.15.70:0/4068875997 wait complete. 2024-09-13T15:44:03.561 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f889a757640 1 -- 172.21.15.70:0/2982608379 shutdown_connections 2024-09-13T15:44:03.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f889a757640 1 --2- 172.21.15.70:0/2982608379 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f889406d3a0 0x7f88940957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f889a757640 1 --2- 172.21.15.70:0/2982608379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889406ca80 0x7f889406ce60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f889a757640 1 -- 172.21.15.70:0/2982608379 >> 172.21.15.70:0/2982608379 conn(0x7f8894098ae0 msgr2=0x7f88940963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f56151ed640 1 -- 172.21.15.70:0/585704942 shutdown_connections 2024-09-13T15:44:03.562 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f889a757640 1 -- 172.21.15.70:0/2982608379 shutdown_connections 2024-09-13T15:44:03.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f889a757640 1 -- 172.21.15.70:0/2982608379 wait complete. 2024-09-13T15:44:03.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.542+0000 7fe8934d0640 1 -- 172.21.15.70:0/704283609 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe88c06b970 msgr2=0x7fe88c06bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.542+0000 7fe8934d0640 1 --2- 172.21.15.70:0/704283609 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe88c06b970 0x7fe88c06bd50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe888009870 tx=0x7fe88802a1c0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.563 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.541+0000 7f56151ed640 1 -- 172.21.15.70:0/585704942 wait complete. 2024-09-13T15:44:03.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.542+0000 7fe8934d0640 1 -- 172.21.15.70:0/704283609 shutdown_connections 2024-09-13T15:44:03.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.542+0000 7fe8934d0640 1 --2- 172.21.15.70:0/704283609 >> v2:172.21.15.70:3300/0 conn(0x7fe88c0957a0 0x7fe88c133740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.542+0000 7fe8934d0640 1 --2- 172.21.15.70:0/704283609 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe88c06b970 0x7fe88c06bd50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.542+0000 7fe8934d0640 1 -- 172.21.15.70:0/704283609 >> 172.21.15.70:0/704283609 conn(0x7fe88c098ae0 msgr2=0x7fe88c0963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.542+0000 7fe8934d0640 1 -- 172.21.15.70:0/704283609 shutdown_connections 2024-09-13T15:44:03.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.542+0000 7fe8934d0640 1 -- 172.21.15.70:0/704283609 wait complete. 2024-09-13T15:44:03.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.542+0000 7fbe1823a640 1 Processor -- start 2024-09-13T15:44:03.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.542+0000 7fbe1823a640 1 -- start start 2024-09-13T15:44:03.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7f889a757640 1 Processor -- start 2024-09-13T15:44:03.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7f56151ed640 1 Processor -- start 2024-09-13T15:44:03.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe1823a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe1012a540 0x7fbe101be450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7f56151ed640 1 -- start start 2024-09-13T15:44:03.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7f889a757640 1 -- start start 2024-09-13T15:44:03.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe1823a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe1012d330 0x7fbe101be990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe1823a640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbe1012dd80 0x7fbe10126e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe1823a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1008b180 con 0x7fbe1012d330 2024-09-13T15:44:03.567 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe1823a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1008b550 con 0x7fbe1012a540 2024-09-13T15:44:03.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe1823a640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1009a120 con 0x7fbe1012dd80 2024-09-13T15:44:03.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe15fd6640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe1012a540 0x7fbe101be450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe167d7640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbe1012dd80 0x7fbe10126e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe15fd6640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe1012a540 0x7fbe101be450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59476/0 (socket says 172.21.15.70:59476) 2024-09-13T15:44:03.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe15fd6640 1 -- 172.21.15.70:0/2900696835 learned_addr learned my addr 172.21.15.70:0/2900696835 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe157d5640 1 --2- 172.21.15.70:0/2900696835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe1012d330 0x7fbe101be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7f889a757640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889406d3a0 0x7f889412bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.568 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7f889a757640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f889406dd10 0x7f889412c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7f889a757640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f889412c660 0x7f88941c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7f889a757640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889412dc90 con 0x7f889412c660 2024-09-13T15:44:03.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7f889a757640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889412cd40 con 0x7f889406d3a0 2024-09-13T15:44:03.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7f889a757640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f889412eba0 con 0x7f889406dd10 2024-09-13T15:44:03.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe157d5640 1 -- 172.21.15.70:0/2900696835 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbe1012dd80 msgr2=0x7fbe10126e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.569 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe157d5640 1 --2- 172.21.15.70:0/2900696835 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fbe1012dd80 0x7fbe10126e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7f56151ed640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f561012a510 0x7f561006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7f56151ed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f561012ddf0 0x7f561006ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe157d5640 1 -- 172.21.15.70:0/2900696835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe1012a540 msgr2=0x7fbe101be450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe157d5640 1 --2- 172.21.15.70:0/2900696835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbe1012a540 0x7fbe101be450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.570 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7f56151ed640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56100715a0 0x7f561006f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.543+0000 7fbe157d5640 1 -- 172.21.15.70:0/2900696835 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe08009a70 con 0x7fbe1012d330 2024-09-13T15:44:03.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f56151ed640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56101344c0 con 0x7f561012ddf0 2024-09-13T15:44:03.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f56151ed640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5610127d30 con 0x7f56100715a0 2024-09-13T15:44:03.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f56151ed640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5610070b30 con 0x7f561012a510 2024-09-13T15:44:03.571 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f560ed9d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f561012a510 0x7f561006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7fe8934d0640 1 Processor -- start 2024-09-13T15:44:03.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f560e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f561012ddf0 0x7f561006ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f560ed9d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f561012a510 0x7f561006ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58698/0 (socket says 172.21.15.70:58698) 2024-09-13T15:44:03.572 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f560ed9d640 1 -- 172.21.15.70:0/1788121039 learned_addr learned my addr 172.21.15.70:0/1788121039 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7fe8934d0640 1 -- start start 2024-09-13T15:44:03.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7fbe157d5640 1 --2- 172.21.15.70:0/2900696835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe1012d330 0x7fbe101be990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000c440 tx=0x7fbe0000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f8893fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889406d3a0 0x7f889412bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f8893fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889406d3a0 0x7f889412bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59486/0 (socket says 172.21.15.70:59486) 2024-09-13T15:44:03.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f8893fff640 1 -- 172.21.15.70:0/2784607085 learned_addr learned my addr 172.21.15.70:0/2784607085 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f88937fe640 1 --2- 172.21.15.70:0/2784607085 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f889406dd10 0x7f889412c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f8898cf4640 1 --2- 172.21.15.70:0/2784607085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f889412c660 0x7f88941c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f560f59e640 1 --2- 172.21.15.70:0/1788121039 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56100715a0 0x7f561006f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.574 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f8893fff640 1 -- 172.21.15.70:0/2784607085 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f889406dd10 msgr2=0x7f889412c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7fbe067fc640 1 -- 172.21.15.70:0/2900696835 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000035f0 con 0x7fbe1012d330 2024-09-13T15:44:03.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7fbe067fc640 1 -- 172.21.15.70:0/2900696835 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe00003770 con 0x7fbe1012d330 2024-09-13T15:44:03.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7fbe067fc640 1 -- 172.21.15.70:0/2900696835 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe00003b20 con 0x7fbe1012d330 2024-09-13T15:44:03.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7fbe1823a640 1 -- 172.21.15.70:0/2900696835 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe101288e0 con 0x7fbe1012d330 2024-09-13T15:44:03.575 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7fe8934d0640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe88c06b970 0x7fe88c08aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.576 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7fbe1823a640 1 -- 172.21.15.70:0/2900696835 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe101beed0 con 0x7fbe1012d330 2024-09-13T15:44:03.576 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7fe8934d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88c06c290 0x7fe88c08b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.576 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f8893fff640 1 --2- 172.21.15.70:0/2784607085 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f889406dd10 0x7f889412c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f560ed9d640 1 -- 172.21.15.70:0/1788121039 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56100715a0 msgr2=0x7f561006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f560ed9d640 1 --2- 172.21.15.70:0/1788121039 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56100715a0 0x7f561006f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f560ed9d640 1 -- 172.21.15.70:0/1788121039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f561012ddf0 msgr2=0x7f561006ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f560ed9d640 1 --2- 172.21.15.70:0/1788121039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f561012ddf0 0x7f561006ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f560ed9d640 1 -- 172.21.15.70:0/1788121039 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56000094d0 con 0x7f561012a510 2024-09-13T15:44:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f560f59e640 1 --2- 172.21.15.70:0/1788121039 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56100715a0 0x7f561006f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:44:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7f560e59c640 1 --2- 172.21.15.70:0/1788121039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f561012ddf0 0x7f561006ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:03.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7fe8934d0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe88c0957a0 0x7fe88c08ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f8893fff640 1 -- 172.21.15.70:0/2784607085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f889412c660 msgr2=0x7f88941c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f8893fff640 1 --2- 172.21.15.70:0/2784607085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f889412c660 0x7f88941c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7fe8934d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c1259e0 con 0x7fe88c06c290 2024-09-13T15:44:03.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7fe890a6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88c06c290 0x7fe88c08b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7fe89126c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe88c06b970 0x7fe88c08aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7fe89126c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe88c06b970 0x7fe88c08aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58716/0 (socket says 172.21.15.70:58716) 2024-09-13T15:44:03.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7fe89126c640 1 -- 172.21.15.70:0/2208278564 learned_addr learned my addr 172.21.15.70:0/2208278564 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f8893fff640 1 -- 172.21.15.70:0/2784607085 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88840094d0 con 0x7f889406d3a0 2024-09-13T15:44:03.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f88937fe640 1 --2- 172.21.15.70:0/2784607085 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f889406dd10 0x7f889412c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:03.579 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.544+0000 7f8898cf4640 1 --2- 172.21.15.70:0/2784607085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f889412c660 0x7f88941c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:03.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7fe8934d0640 1 -- 172.21.15.70:0/2208278564 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c08d130 con 0x7fe88c0957a0 2024-09-13T15:44:03.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7f8893fff640 1 --2- 172.21.15.70:0/2784607085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889406d3a0 0x7f889412bbe0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8884000c00 tx=0x7f88840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7fe891a6d640 1 --2- 172.21.15.70:0/2208278564 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe88c0957a0 0x7fe88c08ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7f8890ff9640 1 -- 172.21.15.70:0/2784607085 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884030b80 con 0x7f889406d3a0 2024-09-13T15:44:03.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7f8890ff9640 1 -- 172.21.15.70:0/2784607085 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8884030ce0 con 0x7f889406d3a0 2024-09-13T15:44:03.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7fe8934d0640 1 -- 172.21.15.70:0/2208278564 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c08c470 con 0x7fe88c06b970 2024-09-13T15:44:03.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7f889a757640 1 -- 172.21.15.70:0/2784607085 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889412ef70 con 0x7f889406d3a0 2024-09-13T15:44:03.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7f889a757640 1 -- 172.21.15.70:0/2784607085 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88941c6f20 con 0x7f889406d3a0 2024-09-13T15:44:03.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7fe890a6b640 1 -- 172.21.15.70:0/2208278564 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe88c06b970 msgr2=0x7fe88c08aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7fe890a6b640 1 --2- 172.21.15.70:0/2208278564 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fe88c06b970 0x7fe88c08aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7fe890a6b640 1 -- 172.21.15.70:0/2208278564 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe88c0957a0 msgr2=0x7fe88c08ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7f560ed9d640 1 --2- 172.21.15.70:0/1788121039 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f561012a510 0x7f561006ea40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f56000090b0 tx=0x7f56000046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7f55f77fe640 1 -- 172.21.15.70:0/1788121039 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600003ba0 con 0x7f561012a510 2024-09-13T15:44:03.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.545+0000 7f55f77fe640 1 -- 172.21.15.70:0/1788121039 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5600003d20 con 0x7f561012a510 2024-09-13T15:44:03.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7fe890a6b640 1 --2- 172.21.15.70:0/2208278564 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe88c0957a0 0x7fe88c08ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7fe890a6b640 1 -- 172.21.15.70:0/2208278564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8880094d0 con 0x7fe88c06c290 2024-09-13T15:44:03.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7fbe067fc640 1 -- 172.21.15.70:0/2900696835 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fbe00003c80 con 0x7fbe1012d330 2024-09-13T15:44:03.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7fbe1823a640 1 -- 172.21.15.70:0/2900696835 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe1012a110 con 0x7fbe1012d330 2024-09-13T15:44:03.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7f55f77fe640 1 -- 172.21.15.70:0/1788121039 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56000327f0 con 0x7f561012a510 2024-09-13T15:44:03.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7f56151ed640 1 -- 172.21.15.70:0/1788121039 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561006fa00 con 0x7f561012a510 2024-09-13T15:44:03.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7fe890a6b640 1 --2- 172.21.15.70:0/2208278564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88c06c290 0x7fe88c08b530 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00c400 tx=0x7fe87c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7fbe067fc640 1 -- 172.21.15.70:0/2900696835 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fbe0005bac0 con 0x7fbe1012d330 2024-09-13T15:44:03.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7f8890ff9640 1 -- 172.21.15.70:0/2784607085 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888403b670 con 0x7f889406d3a0 2024-09-13T15:44:03.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7f887a7fc640 1 -- 172.21.15.70:0/2784607085 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f886c000fc0 con 0x7f889406d3a0 2024-09-13T15:44:03.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7fe891a6d640 1 --2- 172.21.15.70:0/2208278564 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe88c0957a0 0x7fe88c08ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:03.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.547+0000 7fe881ffb640 1 -- 172.21.15.70:0/2208278564 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c0035f0 con 0x7fe88c06c290 2024-09-13T15:44:03.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.547+0000 7fe881ffb640 1 -- 172.21.15.70:0/2208278564 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87c003750 con 0x7fe88c06c290 2024-09-13T15:44:03.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.547+0000 7fe881ffb640 1 -- 172.21.15.70:0/2208278564 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c003b00 con 0x7fe88c06c290 2024-09-13T15:44:03.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7f56151ed640 1 -- 172.21.15.70:0/1788121039 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56101ccfb0 con 0x7f561012a510 2024-09-13T15:44:03.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7fbe067fc640 1 -- 172.21.15.70:0/2900696835 --> v1:172.21.15.70:6812/2095410362 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbdf4051aa0 con 0x7fbdf404ddf0 2024-09-13T15:44:03.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.546+0000 7fbe067fc640 1 -- 172.21.15.70:0/2900696835 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe000602c0 con 0x7fbe1012d330 2024-09-13T15:44:03.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.547+0000 7fe8934d0640 1 -- 172.21.15.70:0/2208278564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88c125290 con 0x7fe88c06c290 2024-09-13T15:44:03.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.547+0000 7f8890ff9640 1 -- 172.21.15.70:0/2784607085 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f8884039040 con 0x7f889406d3a0 2024-09-13T15:44:03.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.547+0000 7fe8934d0640 1 -- 172.21.15.70:0/2208278564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88c1249f0 con 0x7fe88c06c290 2024-09-13T15:44:03.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.548+0000 7f8890ff9640 1 -- 172.21.15.70:0/2784607085 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f888407da80 con 0x7f889406d3a0 2024-09-13T15:44:03.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.548+0000 7fbe067fc640 1 -- 172.21.15.70:0/2900696835 <== osd.3 v1:172.21.15.70:6812/2095410362 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7fbdf4051aa0 con 0x7fbdf404ddf0 2024-09-13T15:44:03.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.548+0000 7f8890ff9640 1 -- 172.21.15.70:0/2784607085 --> v1:172.21.15.70:6800/2246103037 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8880051ad0 con 0x7f888004de20 2024-09-13T15:44:03.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.548+0000 7f56151ed640 1 -- 172.21.15.70:0/1788121039 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f561012c8b0 con 0x7f561012a510 2024-09-13T15:44:03.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.548+0000 7f8890ff9640 1 -- 172.21.15.70:0/2784607085 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f888407de30 con 0x7f889406d3a0 2024-09-13T15:44:03.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.548+0000 7fe8934d0640 1 -- 172.21.15.70:0/2208278564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe88c06c690 con 0x7fe88c06c290 2024-09-13T15:44:03.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.548+0000 7f55f77fe640 1 -- 172.21.15.70:0/1788121039 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f5600039040 con 0x7f561012a510 2024-09-13T15:44:03.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.549+0000 7fe881ffb640 1 -- 172.21.15.70:0/2208278564 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fe87c003c60 con 0x7fe88c06c290 2024-09-13T15:44:03.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.549+0000 7f55f77fe640 1 -- 172.21.15.70:0/1788121039 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f560007ca30 con 0x7f561012a510 2024-09-13T15:44:03.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.549+0000 7f55f77fe640 1 -- 172.21.15.70:0/1788121039 --> v1:172.21.15.70:6808/2115999109 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f55fc051af0 con 0x7f55fc04de40 2024-09-13T15:44:03.590 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.549+0000 7f55f77fe640 1 -- 172.21.15.70:0/1788121039 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f560007eea0 con 0x7f561012a510 2024-09-13T15:44:03.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.549+0000 7fe881ffb640 1 -- 172.21.15.70:0/2208278564 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fe87c05be40 con 0x7fe88c06c290 2024-09-13T15:44:03.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.549+0000 7fe881ffb640 1 -- 172.21.15.70:0/2208278564 --> v1:172.21.15.70:6804/853418053 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe870051b70 con 0x7fe87004dec0 2024-09-13T15:44:03.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.549+0000 7fe881ffb640 1 -- 172.21.15.70:0/2208278564 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe87c05e460 con 0x7fe88c06c290 2024-09-13T15:44:03.591 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.549+0000 7f8890ff9640 1 -- 172.21.15.70:0/2784607085 <== osd.0 v1:172.21.15.70:6800/2246103037 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7f8880051ad0 con 0x7f888004de20 2024-09-13T15:44:03.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.550+0000 7f55f77fe640 1 -- 172.21.15.70:0/1788121039 <== osd.2 v1:172.21.15.70:6808/2115999109 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7f55fc051af0 con 0x7f55fc04de40 2024-09-13T15:44:03.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.550+0000 7fe881ffb640 1 -- 172.21.15.70:0/2208278564 <== osd.1 v1:172.21.15.70:6804/853418053 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7fe870051b70 con 0x7fe87004dec0 2024-09-13T15:44:03.592 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.561+0000 7fbe1823a640 1 -- 172.21.15.70:0/2900696835 --> v1:172.21.15.70:6812/2095410362 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe1009ac70 con 0x7fbdf404ddf0 2024-09-13T15:44:03.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.562+0000 7fbe067fc640 1 -- 172.21.15.70:0/2900696835 <== osd.3 v1:172.21.15.70:6812/2095410362 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3556540785) 0x7fbe1009ac70 con 0x7fbdf404ddf0 2024-09-13T15:44:03.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.562+0000 7fbe1823a640 1 -- 172.21.15.70:0/2900696835 >> v1:172.21.15.70:6812/2095410362 conn(0x7fbdf404ddf0 legacy=0x7fbdf4050230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.562+0000 7fbe1823a640 1 -- 172.21.15.70:0/2900696835 >> v1:172.21.15.70:6816/1229510400 conn(0x7fbdf4047560 legacy=0x7fbdf4049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.562+0000 7fbe1823a640 1 -- 172.21.15.70:0/2900696835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe1012d330 msgr2=0x7fbe101be990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.562+0000 7fbe1823a640 1 --2- 172.21.15.70:0/2900696835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe1012d330 0x7fbe101be990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000c440 tx=0x7fbe0000c840 comp rx=0 tx=0).stop 2024-09-13T15:44:03.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.562+0000 7fbe167d7640 1 -- 172.21.15.70:0/2900696835 reap_dead start 2024-09-13T15:44:03.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.562+0000 7fbe1823a640 1 -- 172.21.15.70:0/2900696835 shutdown_connections 2024-09-13T15:44:03.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.562+0000 7fbe1823a640 1 -- 172.21.15.70:0/2900696835 >> 172.21.15.70:0/2900696835 conn(0x7fbe100a0060 msgr2=0x7fbe10097cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.562+0000 7fbe1823a640 1 -- 172.21.15.70:0/2900696835 shutdown_connections 2024-09-13T15:44:03.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.562+0000 7fbe1823a640 1 -- 172.21.15.70:0/2900696835 wait complete. 2024-09-13T15:44:03.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.563+0000 7f56151ed640 1 -- 172.21.15.70:0/1788121039 --> v1:172.21.15.70:6808/2115999109 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5610134800 con 0x7f55fc04de40 2024-09-13T15:44:03.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.563+0000 7f55f77fe640 1 -- 172.21.15.70:0/1788121039 <== osd.2 v1:172.21.15.70:6808/2115999109 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3324758384) 0x7f5608009990 con 0x7f55fc04de40 2024-09-13T15:44:03.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.564+0000 7f56151ed640 1 -- 172.21.15.70:0/1788121039 >> v1:172.21.15.70:6808/2115999109 conn(0x7f55fc04de40 legacy=0x7f55fc050280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.564+0000 7f56151ed640 1 -- 172.21.15.70:0/1788121039 >> v1:172.21.15.70:6816/1229510400 conn(0x7f55fc0475b0 legacy=0x7f55fc049a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.564+0000 7f56151ed640 1 -- 172.21.15.70:0/1788121039 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f561012a510 msgr2=0x7f561006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.564+0000 7f56151ed640 1 --2- 172.21.15.70:0/1788121039 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f561012a510 0x7f561006ea40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f56000090b0 tx=0x7f56000046c0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.564+0000 7f560f59e640 1 -- 172.21.15.70:0/1788121039 reap_dead start 2024-09-13T15:44:03.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.564+0000 7f56151ed640 1 -- 172.21.15.70:0/1788121039 shutdown_connections 2024-09-13T15:44:03.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.564+0000 7f56151ed640 1 -- 172.21.15.70:0/1788121039 >> 172.21.15.70:0/1788121039 conn(0x7f56100a0060 msgr2=0x7f5610098dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.564+0000 7f56151ed640 1 -- 172.21.15.70:0/1788121039 shutdown_connections 2024-09-13T15:44:03.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.564+0000 7f56151ed640 1 -- 172.21.15.70:0/1788121039 wait complete. 2024-09-13T15:44:03.597 INFO:teuthology.orchestra.run.smithi070.stdout:120259084291 2024-09-13T15:44:03.598 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T15:44:03.599 INFO:teuthology.orchestra.run.smithi070.stdout:115964116995 2024-09-13T15:44:03.599 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T15:44:03.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.578+0000 7f887a7fc640 1 -- 172.21.15.70:0/2784607085 --> v1:172.21.15.70:6800/2246103037 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f886c002cb0 con 0x7f888004de20 2024-09-13T15:44:03.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.579+0000 7f8890ff9640 1 -- 172.21.15.70:0/2784607085 <== osd.0 v1:172.21.15.70:6800/2246103037 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 662925176) 0x7f886c002cb0 con 0x7f888004de20 2024-09-13T15:44:03.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.579+0000 7f887a7fc640 1 -- 172.21.15.70:0/2784607085 >> v1:172.21.15.70:6800/2246103037 conn(0x7f888004de20 legacy=0x7f8880050260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.579+0000 7f887a7fc640 1 -- 172.21.15.70:0/2784607085 >> v1:172.21.15.70:6816/1229510400 conn(0x7f8880047590 legacy=0x7f8880049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.579+0000 7f887a7fc640 1 -- 172.21.15.70:0/2784607085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889406d3a0 msgr2=0x7f889412bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.579+0000 7f887a7fc640 1 --2- 172.21.15.70:0/2784607085 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f889406d3a0 0x7f889412bbe0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8884000c00 tx=0x7f88840046c0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.579+0000 7f8898cf4640 1 -- 172.21.15.70:0/2784607085 reap_dead start 2024-09-13T15:44:03.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.579+0000 7f887a7fc640 1 -- 172.21.15.70:0/2784607085 shutdown_connections 2024-09-13T15:44:03.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.579+0000 7f887a7fc640 1 -- 172.21.15.70:0/2784607085 >> 172.21.15.70:0/2784607085 conn(0x7f8894098ae0 msgr2=0x7f8894096bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.579+0000 7f887a7fc640 1 -- 172.21.15.70:0/2784607085 shutdown_connections 2024-09-13T15:44:03.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.579+0000 7f887a7fc640 1 -- 172.21.15.70:0/2784607085 wait complete. 2024-09-13T15:44:03.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.580+0000 7fe8934d0640 1 -- 172.21.15.70:0/2208278564 --> v1:172.21.15.70:6804/853418053 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe88c060b30 con 0x7fe87004dec0 2024-09-13T15:44:03.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.580+0000 7fe881ffb640 1 -- 172.21.15.70:0/2208278564 <== osd.1 v1:172.21.15.70:6804/853418053 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 2083264769) 0x7fe88c060b30 con 0x7fe87004dec0 2024-09-13T15:44:03.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.580+0000 7fe8934d0640 1 -- 172.21.15.70:0/2208278564 >> v1:172.21.15.70:6804/853418053 conn(0x7fe87004dec0 legacy=0x7fe870050300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.580+0000 7fe8934d0640 1 -- 172.21.15.70:0/2208278564 >> v1:172.21.15.70:6816/1229510400 conn(0x7fe870047630 legacy=0x7fe870049af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.580+0000 7fe8934d0640 1 -- 172.21.15.70:0/2208278564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88c06c290 msgr2=0x7fe88c08b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.580+0000 7fe8934d0640 1 --2- 172.21.15.70:0/2208278564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88c06c290 0x7fe88c08b530 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00c400 tx=0x7fe87c00c8c0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.580+0000 7fe891a6d640 1 -- 172.21.15.70:0/2208278564 reap_dead start 2024-09-13T15:44:03.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.580+0000 7fe8934d0640 1 -- 172.21.15.70:0/2208278564 shutdown_connections 2024-09-13T15:44:03.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.580+0000 7fe8934d0640 1 -- 172.21.15.70:0/2208278564 >> 172.21.15.70:0/2208278564 conn(0x7fe88c098ae0 msgr2=0x7fe88c128f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.581+0000 7fe8934d0640 1 -- 172.21.15.70:0/2208278564 shutdown_connections 2024-09-13T15:44:03.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.581+0000 7fe8934d0640 1 -- 172.21.15.70:0/2208278564 wait complete. 2024-09-13T15:44:03.605 INFO:teuthology.orchestra.run.smithi070.stdout:107374182404 2024-09-13T15:44:03.605 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T15:44:03.606 INFO:teuthology.orchestra.run.smithi070.stdout:111669149700 2024-09-13T15:44:03.606 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T15:44:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f8699219640 1 Processor -- start 2024-09-13T15:44:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f3ce89f4640 1 Processor -- start 2024-09-13T15:44:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f56d3b5e640 1 Processor -- start 2024-09-13T15:44:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f3ce89f4640 1 -- start start 2024-09-13T15:44:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f8699219640 1 -- start start 2024-09-13T15:44:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f56d3b5e640 1 -- start start 2024-09-13T15:44:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f8699219640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f8694134a90 0x7f8694136e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f8699219640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86941373c0 0x7f86941397b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f8699219640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869406e050 con 0x7f86940953c0 2024-09-13T15:44:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f8699219640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869406da90 con 0x7f86941373c0 2024-09-13T15:44:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f946b261640 1 Processor -- start 2024-09-13T15:44:03.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f56d3b5e640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f56cc06ca80 0x7f56cc06ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f56d3b5e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f56cc06dd10 0x7f56cc06d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f56d3b5e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56cc095090 0x7f56cc095500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f56d3b5e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56cc06f1d0 con 0x7f56cc06dd10 2024-09-13T15:44:03.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8692d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f86940953c0 0x7f86940957a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60486/0 (socket says 172.21.15.70:60486) 2024-09-13T15:44:03.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f56d3b5e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc06ebc0 con 0x7f56cc095090 2024-09-13T15:44:03.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f56d3b5e640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc06d8e0 con 0x7f56cc06ca80 2024-09-13T15:44:03.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f3ce89f4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ce412d780 0x7f3ce412db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f3ce89f4640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ce406c8b0 0x7f3ce406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.746 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f3ce89f4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce412d330 con 0x7f3ce406d210 2024-09-13T15:44:03.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f3ce89f4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce406dc70 con 0x7f3ce412d780 2024-09-13T15:44:03.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f3ce89f4640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce406d6c0 con 0x7f3ce406c8b0 2024-09-13T15:44:03.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8692d9d640 1 -- 172.21.15.70:0/2448099836 learned_addr learned my addr 172.21.15.70:0/2448099836 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:03.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.740+0000 7f8699219640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8694095eb0 con 0x7f8694134a90 2024-09-13T15:44:03.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f946b261640 1 -- start start 2024-09-13T15:44:03.747 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f56d18fa640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f56cc06ca80 0x7f56cc06ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f56d10f9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f56cc06dd10 0x7f56cc06d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f56d10f9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f56cc06dd10 0x7f56cc06d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41006/0 (socket says 172.21.15.70:41006) 2024-09-13T15:44:03.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f56d10f9640 1 -- 172.21.15.70:0/628524741 learned_addr learned my addr 172.21.15.70:0/628524741 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f869359e640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86941373c0 0x7f86941397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 -- 172.21.15.70:0/2448099836 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177205475 0 0) 0x7f869406e050 con 0x7f86940953c0 2024-09-13T15:44:03.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f869259c640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f8694134a90 0x7f8694136e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 -- 172.21.15.70:0/2448099836 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f867c003520 con 0x7f86940953c0 2024-09-13T15:44:03.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f3ce259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ce412d780 0x7f3ce412db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f3ce259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ce412d780 0x7f3ce412db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59520/0 (socket says 172.21.15.70:59520) 2024-09-13T15:44:03.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f3ce259c640 1 -- 172.21.15.70:0/3324063676 learned_addr learned my addr 172.21.15.70:0/3324063676 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f3ce1d9b640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ce406c8b0 0x7f3ce406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f56d18fa640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f56cc06ca80 0x7f56cc06ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58728/0 (socket says 172.21.15.70:58728) 2024-09-13T15:44:03.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f56d20fb640 1 --2- 172.21.15.70:0/628524741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56cc095090 0x7f56cc095500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f56d10f9640 1 -- 172.21.15.70:0/628524741 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f56cc06ca80 msgr2=0x7f56cc06ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f56d10f9640 1 --2- 172.21.15.70:0/628524741 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f56cc06ca80 0x7f56cc06ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f56d10f9640 1 -- 172.21.15.70:0/628524741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56cc095090 msgr2=0x7f56cc095500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f56d10f9640 1 --2- 172.21.15.70:0/628524741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56cc095090 0x7f56cc095500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f56d10f9640 1 -- 172.21.15.70:0/628524741 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc097d40 con 0x7f56cc06dd10 2024-09-13T15:44:03.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f946b261640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f946406d2a0 0x7f946406d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f946b261640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f946406dbc0 0x7f946412a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f946b261640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f946412a640 0x7f946412aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f946b261640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946406ecd0 con 0x7f946406dbc0 2024-09-13T15:44:03.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f946b261640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946406bfc0 con 0x7f946412a640 2024-09-13T15:44:03.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 -- 172.21.15.70:0/3324063676 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708011622 0 0) 0x7f3ce412d330 con 0x7f3ce406d210 2024-09-13T15:44:03.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 -- 172.21.15.70:0/3324063676 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ccc003520 con 0x7f3ce406d210 2024-09-13T15:44:03.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f946b261640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f946406e0e0 con 0x7f946406d2a0 2024-09-13T15:44:03.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f94697fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f946412a640 0x7f946412aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f94697fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f946412a640 0x7f946412aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59544/0 (socket says 172.21.15.70:59544) 2024-09-13T15:44:03.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 -- 172.21.15.70:0/2448099836 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1477126526 0 0) 0x7f867c003520 con 0x7f86940953c0 2024-09-13T15:44:03.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 -- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f8694134a90 msgr2=0x7f8694136e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f8694134a90 0x7f8694136e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 -- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86941373c0 msgr2=0x7f86941397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f86941373c0 0x7f86941397b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 -- 172.21.15.70:0/2448099836 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8694097d40 con 0x7f86940953c0 2024-09-13T15:44:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 -- 172.21.15.70:0/2448099836 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2249869809 0 0) 0x7f8684001670 con 0x7f86940953c0 2024-09-13T15:44:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 -- 172.21.15.70:0/2448099836 >> v1:172.21.15.70:6789/0 conn(0x7f86940953c0 legacy=0x7f86940957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f867c0039a0 0x7f867c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c004340 0x7f867c004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f867c005010 0x7f867c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 -- 172.21.15.70:0/2448099836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869406da90 con 0x7f867c004340 2024-09-13T15:44:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 -- 172.21.15.70:0/2448099836 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8694095eb0 con 0x7f867c005010 2024-09-13T15:44:03.753 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.741+0000 7f8691d9b640 1 -- 172.21.15.70:0/2448099836 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f867c00bf70 con 0x7f867c0039a0 2024-09-13T15:44:03.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f869259c640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c004340 0x7f867c004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f8692d9d640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f867c0039a0 0x7f867c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f869359e640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f867c005010 0x7f867c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f94697fe640 1 -- 172.21.15.70:0/73871557 learned_addr learned my addr 172.21.15.70:0/73871557 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f9468ffd640 1 --2- 172.21.15.70:0/73871557 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f946406d2a0 0x7f946406d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 -- 172.21.15.70:0/3324063676 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2030838761 0 0) 0x7f3ccc003520 con 0x7f3ce406d210 2024-09-13T15:44:03.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 -- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ce406c8b0 msgr2=0x7f3ce406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f869259c640 1 -- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f867c0039a0 msgr2=0x7f867c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f869259c640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f867c0039a0 0x7f867c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f869259c640 1 -- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f867c005010 msgr2=0x7f867c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f869259c640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f867c005010 0x7f867c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.755 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f869259c640 1 -- 172.21.15.70:0/2448099836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c003520 con 0x7f867c004340 2024-09-13T15:44:03.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ce406c8b0 0x7f3ce406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 -- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ce412d780 msgr2=0x7f3ce412db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ce412d780 0x7f3ce412db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 -- 172.21.15.70:0/3324063676 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce4067710 con 0x7f3ce406d210 2024-09-13T15:44:03.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 -- 172.21.15.70:0/3324063676 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2249869809 0 0) 0x7f3cdc0019d0 con 0x7f3ce406d210 2024-09-13T15:44:03.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 -- 172.21.15.70:0/3324063676 >> v1:172.21.15.70:6789/0 conn(0x7f3ce406d210 legacy=0x7f3ce4134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ccc003990 0x7f3ccc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.756 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ccc0047c0 0x7f3ccc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ccc005180 0x7f3ccc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 -- 172.21.15.70:0/3324063676 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce406dc70 con 0x7f3ccc0047c0 2024-09-13T15:44:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 -- 172.21.15.70:0/3324063676 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce406d6c0 con 0x7f3ccc005180 2024-09-13T15:44:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce159a640 1 -- 172.21.15.70:0/3324063676 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc00bf70 con 0x7f3ccc003990 2024-09-13T15:44:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce259c640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ce412d780 0x7f3ce412db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:44:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f3ce1d9b640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ce406c8b0 0x7f3ce406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f9463fff640 1 --2- 172.21.15.70:0/73871557 >> v2:172.21.15.70:3300/0 conn(0x7f946406dbc0 0x7f946412a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f94697fe640 1 -- 172.21.15.70:0/73871557 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f946406d2a0 msgr2=0x7f946406d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f94697fe640 1 --2- 172.21.15.70:0/73871557 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f946406d2a0 0x7f946406d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f94697fe640 1 -- 172.21.15.70:0/73871557 >> v2:172.21.15.70:3300/0 conn(0x7f946406dbc0 msgr2=0x7f946412a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f94697fe640 1 --2- 172.21.15.70:0/73871557 >> v2:172.21.15.70:3300/0 conn(0x7f946406dbc0 0x7f946412a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f94697fe640 1 -- 172.21.15.70:0/73871557 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946406f510 con 0x7f946412a640 2024-09-13T15:44:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.742+0000 7f56d10f9640 1 --2- 172.21.15.70:0/628524741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc06dd10 0x7f56cc06d3a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009870 tx=0x7f56bc02a090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460906255cc20421 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f3ce1d9b640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ccc0047c0 0x7f3ccc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f3ce2d9d640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ccc005180 0x7f3ccc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f56d08f8640 1 -- 172.21.15.70:0/628524741 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc02ade0 con 0x7f56cc06dd10 2024-09-13T15:44:03.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f3ce259c640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ccc003990 0x7f3ccc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f56d08f8640 1 -- 172.21.15.70:0/628524741 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc032440 con 0x7f56cc06dd10 2024-09-13T15:44:03.759 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f56d08f8640 1 -- 172.21.15.70:0/628524741 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0327d0 con 0x7f56cc06dd10 2024-09-13T15:44:03.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f3ce2d9d640 1 -- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ccc003990 msgr2=0x7f3ccc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f3ce2d9d640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ccc003990 0x7f3ccc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f3ce2d9d640 1 -- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ccc0047c0 msgr2=0x7f3ccc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:03.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f3ce2d9d640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ccc0047c0 0x7f3ccc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f3ce2d9d640 1 -- 172.21.15.70:0/3324063676 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce4067710 con 0x7f3ccc005180 2024-09-13T15:44:03.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f3ce1d9b640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ccc0047c0 0x7f3ccc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:03.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f869259c640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c004340 0x7f867c004820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8694071ed0 tx=0x7f867c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3527667f86d7992d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f94697fe640 1 --2- 172.21.15.70:0/73871557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f946412a640 0x7f946412aa90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f945c00bf20 tx=0x7f945c0093b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bb024f983d7b0db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f8691d9b640 1 -- 172.21.15.70:0/2448099836 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868802ac70 con 0x7f867c004340 2024-09-13T15:44:03.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f94637fe640 1 -- 172.21.15.70:0/73871557 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00b2e0 con 0x7f946412a640 2024-09-13T15:44:03.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f8691d9b640 1 -- 172.21.15.70:0/2448099836 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868802e070 con 0x7f867c004340 2024-09-13T15:44:03.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f8691d9b640 1 -- 172.21.15.70:0/2448099836 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688034400 con 0x7f867c004340 2024-09-13T15:44:03.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f94637fe640 1 -- 172.21.15.70:0/73871557 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c02d050 con 0x7f946412a640 2024-09-13T15:44:03.762 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.743+0000 7f94637fe640 1 -- 172.21.15.70:0/73871557 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c009840 con 0x7f946412a640 2024-09-13T15:44:03.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f56d3b5e640 1 -- 172.21.15.70:0/628524741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc06dd10 msgr2=0x7f56cc06d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.763 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f56d3b5e640 1 --2- 172.21.15.70:0/628524741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc06dd10 0x7f56cc06d3a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009870 tx=0x7f56bc02a090 comp rx=0 tx=0).stop 2024-09-13T15:44:03.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f56d3b5e640 1 -- 172.21.15.70:0/628524741 shutdown_connections 2024-09-13T15:44:03.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f56d3b5e640 1 --2- 172.21.15.70:0/628524741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56cc095090 0x7f56cc095500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f56d3b5e640 1 --2- 172.21.15.70:0/628524741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc06dd10 0x7f56cc06d3a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f56d3b5e640 1 --2- 172.21.15.70:0/628524741 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f56cc06ca80 0x7f56cc06ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f56d3b5e640 1 -- 172.21.15.70:0/628524741 >> 172.21.15.70:0/628524741 conn(0x7f56cc098ae0 msgr2=0x7f56cc0963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f56d3b5e640 1 -- 172.21.15.70:0/628524741 shutdown_connections 2024-09-13T15:44:03.764 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f56d3b5e640 1 -- 172.21.15.70:0/628524741 wait complete. 2024-09-13T15:44:03.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f3ce2d9d640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ccc005180 0x7f3ccc00b980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c520 tx=0x7f3ccc0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9862b73c104f1332 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f3ce159a640 1 -- 172.21.15.70:0/3324063676 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc003b70 con 0x7f3ccc005180 2024-09-13T15:44:03.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f3ce159a640 1 -- 172.21.15.70:0/3324063676 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cdc003cd0 con 0x7f3ccc005180 2024-09-13T15:44:03.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f3ce159a640 1 -- 172.21.15.70:0/3324063676 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc005330 con 0x7f3ccc005180 2024-09-13T15:44:03.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f8699219640 1 -- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c004340 msgr2=0x7f867c004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.765 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f8699219640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c004340 0x7f867c004820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8694071ed0 tx=0x7f867c0057f0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f8699219640 1 -- 172.21.15.70:0/2448099836 shutdown_connections 2024-09-13T15:44:03.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f8699219640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f867c005010 0x7f867c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f8699219640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c004340 0x7f867c004820 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f8699219640 1 --2- 172.21.15.70:0/2448099836 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f867c0039a0 0x7f867c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f946b261640 1 -- 172.21.15.70:0/73871557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f946412a640 msgr2=0x7f946412aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f946b261640 1 --2- 172.21.15.70:0/73871557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f946412a640 0x7f946412aa90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f945c00bf20 tx=0x7f945c0093b0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f8699219640 1 -- 172.21.15.70:0/2448099836 >> 172.21.15.70:0/2448099836 conn(0x7f8694098ae0 msgr2=0x7f86940963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f8699219640 1 -- 172.21.15.70:0/2448099836 shutdown_connections 2024-09-13T15:44:03.767 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f8699219640 1 -- 172.21.15.70:0/2448099836 wait complete. 2024-09-13T15:44:03.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f946b261640 1 -- 172.21.15.70:0/73871557 shutdown_connections 2024-09-13T15:44:03.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f946b261640 1 --2- 172.21.15.70:0/73871557 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f946412a640 0x7f946412aa90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f946b261640 1 --2- 172.21.15.70:0/73871557 >> v2:172.21.15.70:3300/0 conn(0x7f946406dbc0 0x7f946412a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f946b261640 1 --2- 172.21.15.70:0/73871557 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f946406d2a0 0x7f946406d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f946b261640 1 -- 172.21.15.70:0/73871557 >> 172.21.15.70:0/73871557 conn(0x7f946409cde0 msgr2=0x7f946409d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.744+0000 7f946b261640 1 -- 172.21.15.70:0/73871557 shutdown_connections 2024-09-13T15:44:03.768 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f946b261640 1 -- 172.21.15.70:0/73871557 wait complete. 2024-09-13T15:44:03.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f56d3b5e640 1 Processor -- start 2024-09-13T15:44:03.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f56d3b5e640 1 -- start start 2024-09-13T15:44:03.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ccc005180 msgr2=0x7f3ccc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f3ce89f4640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ccc005180 0x7f3ccc00b980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c520 tx=0x7f3ccc0057f0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3324063676 shutdown_connections 2024-09-13T15:44:03.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f3ce89f4640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ccc005180 0x7f3ccc00b980 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f3ce89f4640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ccc0047c0 0x7f3ccc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f3ce89f4640 1 --2- 172.21.15.70:0/3324063676 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ccc003990 0x7f3ccc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3324063676 >> 172.21.15.70:0/3324063676 conn(0x7f3ce40a02e0 msgr2=0x7f3ce409c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3324063676 shutdown_connections 2024-09-13T15:44:03.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3324063676 wait complete. 2024-09-13T15:44:03.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f946b261640 1 Processor -- start 2024-09-13T15:44:03.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f56d3b5e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56cc06ca80 0x7f56cc1254b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f56d3b5e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc06dd10 0x7f56cc1259f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.771 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.745+0000 7f946b261640 1 -- start start 2024-09-13T15:44:03.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f8699219640 1 Processor -- start 2024-09-13T15:44:03.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d18fa640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56cc06ca80 0x7f56cc1254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d3b5e640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f56cc095090 0x7f56cc125f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d18fa640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56cc06ca80 0x7f56cc1254b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59572/0 (socket says 172.21.15.70:59572) 2024-09-13T15:44:03.772 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d10f9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc06dd10 0x7f56cc1259f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f8699219640 1 -- start start 2024-09-13T15:44:03.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d10f9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc06dd10 0x7f56cc1259f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41044/0 (socket says 172.21.15.70:41044) 2024-09-13T15:44:03.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d20fb640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f56cc095090 0x7f56cc125f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d20fb640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f56cc095090 0x7f56cc125f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58774/0 (socket says 172.21.15.70:58774) 2024-09-13T15:44:03.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d3b5e640 1 -- 172.21.15.70:0/1353788546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc071c30 con 0x7f56cc06dd10 2024-09-13T15:44:03.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d18fa640 1 -- 172.21.15.70:0/1353788546 learned_addr learned my addr 172.21.15.70:0/1353788546 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d3b5e640 1 -- 172.21.15.70:0/1353788546 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc1332b0 con 0x7f56cc06ca80 2024-09-13T15:44:03.773 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d3b5e640 1 -- 172.21.15.70:0/1353788546 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc12c500 con 0x7f56cc095090 2024-09-13T15:44:03.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f946b261640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f946406d2a0 0x7f94641be250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f8699219640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f867c0039a0 0x7f86941294f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f8699219640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f867c004340 0x7f8694129a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f8699219640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c005010 0x7f86941c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f8699219640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869412c1a0 con 0x7f867c005010 2024-09-13T15:44:03.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f8699219640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8694071b50 con 0x7f867c0039a0 2024-09-13T15:44:03.774 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f8699219640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f869412c740 con 0x7f867c004340 2024-09-13T15:44:03.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f8692d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f867c0039a0 0x7f86941294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f869359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c005010 0x7f86941c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f8692d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f867c0039a0 0x7f86941294f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:59580/0 (socket says 172.21.15.70:59580) 2024-09-13T15:44:03.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f8692d9d640 1 -- 172.21.15.70:0/2270213950 learned_addr learned my addr 172.21.15.70:0/2270213950 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d10f9640 1 -- 172.21.15.70:0/1353788546 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f56cc095090 msgr2=0x7f56cc125f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d10f9640 1 --2- 172.21.15.70:0/1353788546 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f56cc095090 0x7f56cc125f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d10f9640 1 -- 172.21.15.70:0/1353788546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56cc06ca80 msgr2=0x7f56cc1254b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d10f9640 1 --2- 172.21.15.70:0/1353788546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56cc06ca80 0x7f56cc1254b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.775 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d10f9640 1 -- 172.21.15.70:0/1353788546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc0094d0 con 0x7f56cc06dd10 2024-09-13T15:44:03.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f869259c640 1 --2- 172.21.15.70:0/2270213950 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f867c004340 0x7f8694129a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.746+0000 7f56d20fb640 1 --2- 172.21.15.70:0/1353788546 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f56cc095090 0x7f56cc125f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:03.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f3ce89f4640 1 Processor -- start 2024-09-13T15:44:03.776 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f3ce89f4640 1 -- start start 2024-09-13T15:44:03.777 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f56d10f9640 1 --2- 172.21.15.70:0/1353788546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc06dd10 0x7f56cc1259f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f56bc031000 tx=0x7f56bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.777 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f869359e640 1 -- 172.21.15.70:0/2270213950 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f867c004340 msgr2=0x7f8694129a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.777 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f869359e640 1 --2- 172.21.15.70:0/2270213950 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f867c004340 0x7f8694129a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.777 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f869359e640 1 -- 172.21.15.70:0/2270213950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f867c0039a0 msgr2=0x7f86941294f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.777 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f869359e640 1 --2- 172.21.15.70:0/2270213950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f867c0039a0 0x7f86941294f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.777 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f869359e640 1 -- 172.21.15.70:0/2270213950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869412ace0 con 0x7f867c005010 2024-09-13T15:44:03.778 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f869259c640 1 --2- 172.21.15.70:0/2270213950 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f867c004340 0x7f8694129a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:03.778 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f946b261640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f946406dbc0 0x7f94641be790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.778 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f9468ffd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f946406d2a0 0x7f94641be250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.778 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f946b261640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f946412a640 0x7f946408aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.778 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f946b261640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946408d530 con 0x7f946412a640 2024-09-13T15:44:03.778 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f8692d9d640 1 --2- 172.21.15.70:0/2270213950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f867c0039a0 0x7f86941294f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:03.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f56c27fc640 1 -- 172.21.15.70:0/1353788546 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc03d850 con 0x7f56cc06dd10 2024-09-13T15:44:03.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f56c27fc640 1 -- 172.21.15.70:0/1353788546 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc03d9f0 con 0x7f56cc06dd10 2024-09-13T15:44:03.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f56c27fc640 1 -- 172.21.15.70:0/1353788546 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc03dda0 con 0x7f56cc06dd10 2024-09-13T15:44:03.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f946b261640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946408e1f0 con 0x7f946406dbc0 2024-09-13T15:44:03.779 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f946b261640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f946408d1d0 con 0x7f946406d2a0 2024-09-13T15:44:03.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f56d3b5e640 1 -- 172.21.15.70:0/1353788546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56cc12c130 con 0x7f56cc06dd10 2024-09-13T15:44:03.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f9468ffd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f946406d2a0 0x7f94641be250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58784/0 (socket says 172.21.15.70:58784) 2024-09-13T15:44:03.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f9468ffd640 1 -- 172.21.15.70:0/2468794807 learned_addr learned my addr 172.21.15.70:0/2468794807 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f94697fe640 1 --2- 172.21.15.70:0/2468794807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f946412a640 0x7f946408aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f9463fff640 1 --2- 172.21.15.70:0/2468794807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f946406dbc0 0x7f94641be790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f56d3b5e640 1 -- 172.21.15.70:0/1353788546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc126650 con 0x7f56cc06dd10 2024-09-13T15:44:03.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f3ce89f4640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ccc003990 0x7f3ce4127f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f3ce259c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ccc003990 0x7f3ce4127f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.747+0000 7f3ce259c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ccc003990 0x7f3ce4127f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:58800/0 (socket says 172.21.15.70:58800) 2024-09-13T15:44:03.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f9468ffd640 1 -- 172.21.15.70:0/2468794807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f946406dbc0 msgr2=0x7f94641be790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f9468ffd640 1 --2- 172.21.15.70:0/2468794807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f946406dbc0 0x7f94641be790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f9468ffd640 1 -- 172.21.15.70:0/2468794807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f946412a640 msgr2=0x7f946408aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f9468ffd640 1 --2- 172.21.15.70:0/2468794807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f946412a640 0x7f946408aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f3ce89f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ccc0047c0 0x7f3ce41284c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f9468ffd640 1 -- 172.21.15.70:0/2468794807 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c00bb50 con 0x7f946406d2a0 2024-09-13T15:44:03.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f3ce259c640 1 -- 172.21.15.70:0/3364477043 learned_addr learned my addr 172.21.15.70:0/3364477043 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:03.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f3ce89f4640 1 --2- 172.21.15.70:0/3364477043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ccc005180 0x7f3ce4128a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:03.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f3ce2d9d640 1 --2- 172.21.15.70:0/3364477043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ccc005180 0x7f3ce4128a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f3ce1d9b640 1 --2- 172.21.15.70:0/3364477043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ccc0047c0 0x7f3ce41284c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:03.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3364477043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce408b690 con 0x7f3ccc0047c0 2024-09-13T15:44:03.783 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f9468ffd640 1 --2- 172.21.15.70:0/2468794807 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f946406d2a0 0x7f94641be250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9458012fc0 tx=0x7f9458010490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f869359e640 1 --2- 172.21.15.70:0/2270213950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c005010 0x7f86941c65c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8694099880 tx=0x7f868c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f94617fa640 1 -- 172.21.15.70:0/2468794807 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945800db10 con 0x7f946406d2a0 2024-09-13T15:44:03.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3364477043 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce412abf0 con 0x7f3ccc005180 2024-09-13T15:44:03.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3364477043 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce412a3f0 con 0x7f3ccc003990 2024-09-13T15:44:03.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f867b7fe640 1 -- 172.21.15.70:0/2270213950 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c00e070 con 0x7f867c005010 2024-09-13T15:44:03.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f946b261640 1 -- 172.21.15.70:0/2468794807 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94641becd0 con 0x7f946406d2a0 2024-09-13T15:44:03.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f946b261640 1 -- 172.21.15.70:0/2468794807 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946408b530 con 0x7f946406d2a0 2024-09-13T15:44:03.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f94617fa640 1 -- 172.21.15.70:0/2468794807 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9458013040 con 0x7f946406d2a0 2024-09-13T15:44:03.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f867b7fe640 1 -- 172.21.15.70:0/2270213950 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c011e50 con 0x7f867c005010 2024-09-13T15:44:03.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f8699219640 1 -- 172.21.15.70:0/2270213950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8694129f70 con 0x7f867c005010 2024-09-13T15:44:03.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.749+0000 7f8699219640 1 -- 172.21.15.70:0/2270213950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86941c6ce0 con 0x7f867c005010 2024-09-13T15:44:03.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f56c27fc640 1 -- 172.21.15.70:0/1353788546 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f56bc039040 con 0x7f56cc06dd10 2024-09-13T15:44:03.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.749+0000 7f56d3b5e640 1 -- 172.21.15.70:0/1353788546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56cc12b9a0 con 0x7f56cc06dd10 2024-09-13T15:44:03.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.749+0000 7f867b7fe640 1 -- 172.21.15.70:0/2270213950 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c003990 con 0x7f867c005010 2024-09-13T15:44:03.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.749+0000 7f56c27fc640 1 -- 172.21.15.70:0/1353788546 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f56bc07d0b0 con 0x7f56cc06dd10 2024-09-13T15:44:03.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.749+0000 7f94617fa640 1 -- 172.21.15.70:0/2468794807 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945800d2e0 con 0x7f946406d2a0 2024-09-13T15:44:03.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f3ce259c640 1 -- 172.21.15.70:0/3364477043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ccc005180 msgr2=0x7f3ce4128a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f3ce259c640 1 --2- 172.21.15.70:0/3364477043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ccc005180 0x7f3ce4128a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f3ce259c640 1 -- 172.21.15.70:0/3364477043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ccc0047c0 msgr2=0x7f3ce41284c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f3ce259c640 1 --2- 172.21.15.70:0/3364477043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ccc0047c0 0x7f3ce41284c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.748+0000 7f3ce259c640 1 -- 172.21.15.70:0/3364477043 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce409d550 con 0x7f3ccc003990 2024-09-13T15:44:03.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.749+0000 7f3ce259c640 1 --2- 172.21.15.70:0/3364477043 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ccc003990 0x7f3ce4127f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4011510 tx=0x7f3cd4009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:03.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.749+0000 7f3cd2ffd640 1 -- 172.21.15.70:0/3364477043 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd400c070 con 0x7f3ccc003990 2024-09-13T15:44:03.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.749+0000 7f3cd2ffd640 1 -- 172.21.15.70:0/3364477043 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd4015870 con 0x7f3ccc003990 2024-09-13T15:44:03.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.749+0000 7f3cd2ffd640 1 -- 172.21.15.70:0/3364477043 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4015c20 con 0x7f3ccc003990 2024-09-13T15:44:03.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.749+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3364477043 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce41c21c0 con 0x7f3ccc003990 2024-09-13T15:44:03.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.749+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3364477043 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce41c2620 con 0x7f3ccc003990 2024-09-13T15:44:03.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.750+0000 7f946b261640 1 -- 172.21.15.70:0/2468794807 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9444005250 con 0x7f946406d2a0 2024-09-13T15:44:03.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.750+0000 7f3ce2d9d640 1 --2- 172.21.15.70:0/3364477043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ccc005180 0x7f3ce4128a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:44:03.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.750+0000 7f8699219640 1 -- 172.21.15.70:0/2270213950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8694095b60 con 0x7f867c005010 2024-09-13T15:44:03.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.750+0000 7f94617fa640 1 -- 172.21.15.70:0/2468794807 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f9458005990 con 0x7f946406d2a0 2024-09-13T15:44:03.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.751+0000 7f94617fa640 1 -- 172.21.15.70:0/2468794807 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f9458060c90 con 0x7f946406d2a0 2024-09-13T15:44:03.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.751+0000 7f3cd2ffd640 1 -- 172.21.15.70:0/3364477043 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f3cd4013400 con 0x7f3ccc003990 2024-09-13T15:44:03.789 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.751+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3364477043 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc8005250 con 0x7f3ccc003990 2024-09-13T15:44:03.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.751+0000 7f867b7fe640 1 -- 172.21.15.70:0/2270213950 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f868c003af0 con 0x7f867c005010 2024-09-13T15:44:03.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.751+0000 7f3cd2ffd640 1 -- 172.21.15.70:0/3364477043 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f3cd405cb40 con 0x7f3ccc003990 2024-09-13T15:44:03.790 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.752+0000 7f867b7fe640 1 -- 172.21.15.70:0/2270213950 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f868c05d110 con 0x7f867c005010 2024-09-13T15:44:03.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.754+0000 7f56c27fc640 1 -- 172.21.15.70:0/1353788546 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f56bc047d00 con 0x7f56cc06dd10 2024-09-13T15:44:03.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.755+0000 7f94617fa640 1 -- 172.21.15.70:0/2468794807 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f945803e860 con 0x7f946406d2a0 2024-09-13T15:44:03.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.757+0000 7f867b7fe640 1 -- 172.21.15.70:0/2270213950 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f868c03ad60 con 0x7f867c005010 2024-09-13T15:44:03.791 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.758+0000 7f3cd2ffd640 1 -- 172.21.15.70:0/3364477043 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f3cd403a830 con 0x7f3ccc003990 2024-09-13T15:44:03.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.922+0000 7f8699219640 1 -- 172.21.15.70:0/2270213950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f869406cdd0 con 0x7f867c005010 2024-09-13T15:44:03.924 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.922+0000 7f867b7fe640 1 -- 172.21.15.70:0/2270213950 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f868c03e380 con 0x7f867c005010 2024-09-13T15:44:03.924 INFO:teuthology.orchestra.run.smithi070.stdout:120259084290 2024-09-13T15:44:03.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.924+0000 7f8699219640 1 -- 172.21.15.70:0/2270213950 >> v1:172.21.15.70:6816/1229510400 conn(0x7f86800475b0 legacy=0x7f8680049a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.924+0000 7f8699219640 1 -- 172.21.15.70:0/2270213950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c005010 msgr2=0x7f86941c65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.924+0000 7f8699219640 1 --2- 172.21.15.70:0/2270213950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c005010 0x7f86941c65c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8694099880 tx=0x7f868c0115a0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.924+0000 7f8699219640 1 -- 172.21.15.70:0/2270213950 shutdown_connections 2024-09-13T15:44:03.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.924+0000 7f8699219640 1 --2- 172.21.15.70:0/2270213950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c005010 0x7f86941c65c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.924+0000 7f8699219640 1 --2- 172.21.15.70:0/2270213950 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f867c004340 0x7f8694129a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.926 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.924+0000 7f8699219640 1 --2- 172.21.15.70:0/2270213950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f867c0039a0 0x7f86941294f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.924+0000 7f8699219640 1 -- 172.21.15.70:0/2270213950 >> 172.21.15.70:0/2270213950 conn(0x7f8694098ae0 msgr2=0x7f86941353b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.924+0000 7f8699219640 1 -- 172.21.15.70:0/2270213950 shutdown_connections 2024-09-13T15:44:03.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.924+0000 7f8699219640 1 -- 172.21.15.70:0/2270213950 wait complete. 2024-09-13T15:44:03.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.925+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3364477043 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3cc80057d0 con 0x7f3ccc003990 2024-09-13T15:44:03.927 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.925+0000 7f3cd2ffd640 1 -- 172.21.15.70:0/3364477043 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3cd4019030 con 0x7f3ccc003990 2024-09-13T15:44:03.927 INFO:teuthology.orchestra.run.smithi070.stdout:107374182403 2024-09-13T15:44:03.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.927+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3364477043 >> v1:172.21.15.70:6816/1229510400 conn(0x7f3cc404fdc0 legacy=0x7f3cc4052280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.927+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3364477043 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ccc003990 msgr2=0x7f3ce4127f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.927+0000 7f3ce89f4640 1 --2- 172.21.15.70:0/3364477043 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ccc003990 0x7f3ce4127f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4011510 tx=0x7f3cd4009f90 comp rx=0 tx=0).stop 2024-09-13T15:44:03.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.927+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3364477043 shutdown_connections 2024-09-13T15:44:03.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.927+0000 7f3ce89f4640 1 --2- 172.21.15.70:0/3364477043 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ccc005180 0x7f3ce4128a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.929 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.927+0000 7f3ce89f4640 1 --2- 172.21.15.70:0/3364477043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ccc0047c0 0x7f3ce41284c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.927+0000 7f3ce89f4640 1 --2- 172.21.15.70:0/3364477043 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ccc003990 0x7f3ce4127f80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.927+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3364477043 >> 172.21.15.70:0/3364477043 conn(0x7f3ce40a02e0 msgr2=0x7f3ce41b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.927+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3364477043 shutdown_connections 2024-09-13T15:44:03.930 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.927+0000 7f3ce89f4640 1 -- 172.21.15.70:0/3364477043 wait complete. 2024-09-13T15:44:03.941 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182404 got 107374182403 for osd.0 2024-09-13T15:44:03.942 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084291 got 120259084290 for osd.3 2024-09-13T15:44:03.946 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.944+0000 7f946b261640 1 -- 172.21.15.70:0/2468794807 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9444005500 con 0x7f946406d2a0 2024-09-13T15:44:03.947 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.945+0000 7f94617fa640 1 -- 172.21.15.70:0/2468794807 <== mon.2 v2:172.21.15.70:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9458041e80 con 0x7f946406d2a0 2024-09-13T15:44:03.947 INFO:teuthology.orchestra.run.smithi070.stdout:115964116994 2024-09-13T15:44:03.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.946+0000 7f946b261640 1 -- 172.21.15.70:0/2468794807 >> v1:172.21.15.70:6816/1229510400 conn(0x7f9448047560 legacy=0x7f9448049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.946+0000 7f946b261640 1 -- 172.21.15.70:0/2468794807 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f946406d2a0 msgr2=0x7f94641be250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.946+0000 7f946b261640 1 --2- 172.21.15.70:0/2468794807 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f946406d2a0 0x7f94641be250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9458012fc0 tx=0x7f9458010490 comp rx=0 tx=0).stop 2024-09-13T15:44:03.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.947+0000 7f946b261640 1 -- 172.21.15.70:0/2468794807 shutdown_connections 2024-09-13T15:44:03.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.947+0000 7f946b261640 1 --2- 172.21.15.70:0/2468794807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f946412a640 0x7f946408aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.947+0000 7f946b261640 1 --2- 172.21.15.70:0/2468794807 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f946406dbc0 0x7f94641be790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.947+0000 7f946b261640 1 --2- 172.21.15.70:0/2468794807 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f946406d2a0 0x7f94641be250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.947+0000 7f946b261640 1 -- 172.21.15.70:0/2468794807 >> 172.21.15.70:0/2468794807 conn(0x7f946409cde0 msgr2=0x7f946406ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.947+0000 7f946b261640 1 -- 172.21.15.70:0/2468794807 shutdown_connections 2024-09-13T15:44:03.950 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.947+0000 7f946b261640 1 -- 172.21.15.70:0/2468794807 wait complete. 2024-09-13T15:44:03.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.948+0000 7f56d3b5e640 1 -- 172.21.15.70:0/1353788546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f56cc12a7a0 con 0x7f56cc06dd10 2024-09-13T15:44:03.952 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.948+0000 7f56c27fc640 1 -- 172.21.15.70:0/1353788546 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f56bc037090 con 0x7f56cc06dd10 2024-09-13T15:44:03.952 INFO:teuthology.orchestra.run.smithi070.stdout:111669149699 2024-09-13T15:44:03.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.950+0000 7f56d3b5e640 1 -- 172.21.15.70:0/1353788546 >> v1:172.21.15.70:6816/1229510400 conn(0x7f56b0047630 legacy=0x7f56b0049af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.950+0000 7f56d3b5e640 1 -- 172.21.15.70:0/1353788546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc06dd10 msgr2=0x7f56cc1259f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:03.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.950+0000 7f56d3b5e640 1 --2- 172.21.15.70:0/1353788546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc06dd10 0x7f56cc1259f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f56bc031000 tx=0x7f56bc0046c0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.953 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.950+0000 7f56d3b5e640 1 -- 172.21.15.70:0/1353788546 shutdown_connections 2024-09-13T15:44:03.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.950+0000 7f56d3b5e640 1 --2- 172.21.15.70:0/1353788546 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f56cc095090 0x7f56cc125f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.950+0000 7f56d3b5e640 1 --2- 172.21.15.70:0/1353788546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc06dd10 0x7f56cc1259f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.950+0000 7f56d3b5e640 1 --2- 172.21.15.70:0/1353788546 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56cc06ca80 0x7f56cc1254b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:03.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.950+0000 7f56d3b5e640 1 -- 172.21.15.70:0/1353788546 >> 172.21.15.70:0/1353788546 conn(0x7f56cc098ae0 msgr2=0x7f56cc096bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:03.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.950+0000 7f56d3b5e640 1 -- 172.21.15.70:0/1353788546 shutdown_connections 2024-09-13T15:44:03.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:03.950+0000 7f56d3b5e640 1 -- 172.21.15.70:0/1353788546 wait complete. 2024-09-13T15:44:03.961 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964116995 got 115964116994 for osd.2 2024-09-13T15:44:03.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 111669149700 got 111669149699 for osd.1 2024-09-13T15:44:04.942 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T15:44:04.943 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T15:44:04.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T15:44:04.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T15:44:05.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.055+0000 7f6a8163a640 1 Processor -- start 2024-09-13T15:44:05.056 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.055+0000 7ff8c5cd0640 1 Processor -- start 2024-09-13T15:44:05.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.055+0000 7f6a8163a640 1 -- start start 2024-09-13T15:44:05.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.055+0000 7ff8c5cd0640 1 -- start start 2024-09-13T15:44:05.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.055+0000 7ff8c5cd0640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff8c012d330 0x7ff8c012d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.055+0000 7ff8c5cd0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8c012dd80 0x7ff8c0129e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.057 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.055+0000 7ff8c5cd0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff8c012a370 0x7ff8c012a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.055+0000 7f6a8163a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a7c06d2a0 0x7f6a7c06d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.055+0000 7f6a8163a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6a7c06dbc0 0x7f6a7c12a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.055+0000 7ff8c5cd0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c006d5d0 con 0x7ff8c012a370 2024-09-13T15:44:05.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.055+0000 7ff8c5cd0640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c00971b0 con 0x7ff8c012dd80 2024-09-13T15:44:05.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.055+0000 7ff8c5cd0640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c00979f0 con 0x7ff8c012d330 2024-09-13T15:44:05.058 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7f6a8163a640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6a7c12a640 0x7f6a7c12aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7f6a8163a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c06ecd0 con 0x7f6a7c06dbc0 2024-09-13T15:44:05.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7f6a8163a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c06bfc0 con 0x7f6a7c06d2a0 2024-09-13T15:44:05.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7f6a8163a640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c06e0e0 con 0x7f6a7c12a640 2024-09-13T15:44:05.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7ff8beffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8c012dd80 0x7ff8c0129e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7ff8beffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8c012dd80 0x7ff8c0129e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33038/0 (socket says 172.21.15.70:33038) 2024-09-13T15:44:05.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7ff8beffd640 1 -- 172.21.15.70:0/2583761352 learned_addr learned my addr 172.21.15.70:0/2583761352 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:05.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7ff8bf7fe640 1 --2- 172.21.15.70:0/2583761352 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff8c012d330 0x7ff8c012d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.059 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7ff8bffff640 1 --2- 172.21.15.70:0/2583761352 >> v2:172.21.15.70:3300/0 conn(0x7ff8c012a370 0x7ff8c012a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7f6a7a7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6a7c06dbc0 0x7f6a7c12a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7f6a7affd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a7c06d2a0 0x7f6a7c06d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7f6a7a7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6a7c06dbc0 0x7f6a7c12a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53972/0 (socket says 172.21.15.70:53972) 2024-09-13T15:44:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7f6a7a7fc640 1 -- 172.21.15.70:0/3230314440 learned_addr learned my addr 172.21.15.70:0/3230314440 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7ff8beffd640 1 -- 172.21.15.70:0/2583761352 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff8c012d330 msgr2=0x7ff8c012d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7ff8beffd640 1 --2- 172.21.15.70:0/2583761352 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff8c012d330 0x7ff8c012d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7ff8beffd640 1 -- 172.21.15.70:0/2583761352 >> v2:172.21.15.70:3300/0 conn(0x7ff8c012a370 msgr2=0x7ff8c012a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7ff8beffd640 1 --2- 172.21.15.70:0/2583761352 >> v2:172.21.15.70:3300/0 conn(0x7ff8c012a370 0x7ff8c012a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7ff8beffd640 1 -- 172.21.15.70:0/2583761352 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c012e1a0 con 0x7ff8c012dd80 2024-09-13T15:44:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7ff8bffff640 1 --2- 172.21.15.70:0/2583761352 >> v2:172.21.15.70:3300/0 conn(0x7ff8c012a370 0x7ff8c012a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:44:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7f6a7b7fe640 1 --2- 172.21.15.70:0/3230314440 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6a7c12a640 0x7f6a7c12aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7ff8bf7fe640 1 --2- 172.21.15.70:0/2583761352 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff8c012d330 0x7ff8c012d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:44:05.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7f6a7affd640 1 -- 172.21.15.70:0/3230314440 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6a7c12a640 msgr2=0x7f6a7c12aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:05.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7f6a7affd640 1 --2- 172.21.15.70:0/3230314440 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6a7c12a640 0x7f6a7c12aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.062 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7f6a7affd640 1 -- 172.21.15.70:0/3230314440 >> v2:172.21.15.70:3300/0 conn(0x7f6a7c06dbc0 msgr2=0x7f6a7c12a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:05.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7f6a7affd640 1 --2- 172.21.15.70:0/3230314440 >> v2:172.21.15.70:3300/0 conn(0x7f6a7c06dbc0 0x7f6a7c12a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7f6a7affd640 1 -- 172.21.15.70:0/3230314440 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c06f510 con 0x7f6a7c06d2a0 2024-09-13T15:44:05.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.056+0000 7f6a7a7fc640 1 --2- 172.21.15.70:0/3230314440 >> v2:172.21.15.70:3300/0 conn(0x7f6a7c06dbc0 0x7f6a7c12a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:05.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.057+0000 7ff8beffd640 1 --2- 172.21.15.70:0/2583761352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8c012dd80 0x7ff8c0129e30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40098a0 tx=0x7ff8b402a0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=800d60a9dc56fc63 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:05.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.057+0000 7ff8be7fc640 1 -- 172.21.15.70:0/2583761352 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b402add0 con 0x7ff8c012dd80 2024-09-13T15:44:05.063 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.057+0000 7ff8be7fc640 1 -- 172.21.15.70:0/2583761352 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b4032440 con 0x7ff8c012dd80 2024-09-13T15:44:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.057+0000 7ff8be7fc640 1 -- 172.21.15.70:0/2583761352 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b40327d0 con 0x7ff8c012dd80 2024-09-13T15:44:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.058+0000 7f6a7affd640 1 --2- 172.21.15.70:0/3230314440 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a7c06d2a0 0x7f6a7c06d680 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0097d0 tx=0x7f6a6c02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cba4de48bb6abbd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.058+0000 7f6a79ffb640 1 -- 172.21.15.70:0/3230314440 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c02ad90 con 0x7f6a7c06d2a0 2024-09-13T15:44:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.058+0000 7f6a79ffb640 1 -- 172.21.15.70:0/3230314440 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c032440 con 0x7f6a7c06d2a0 2024-09-13T15:44:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.058+0000 7f6a79ffb640 1 -- 172.21.15.70:0/3230314440 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c0327d0 con 0x7f6a7c06d2a0 2024-09-13T15:44:05.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.058+0000 7ff8c5cd0640 1 -- 172.21.15.70:0/2583761352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8c012dd80 msgr2=0x7ff8c0129e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.058+0000 7ff8c5cd0640 1 --2- 172.21.15.70:0/2583761352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8c012dd80 0x7ff8c0129e30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40098a0 tx=0x7ff8b402a0c0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.058+0000 7ff8c5cd0640 1 -- 172.21.15.70:0/2583761352 shutdown_connections 2024-09-13T15:44:05.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.058+0000 7ff8c5cd0640 1 --2- 172.21.15.70:0/2583761352 >> v2:172.21.15.70:3300/0 conn(0x7ff8c012a370 0x7ff8c012a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.058+0000 7ff8c5cd0640 1 --2- 172.21.15.70:0/2583761352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8c012dd80 0x7ff8c0129e30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.058+0000 7ff8c5cd0640 1 --2- 172.21.15.70:0/2583761352 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff8c012d330 0x7ff8c012d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.065 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.058+0000 7ff8c5cd0640 1 -- 172.21.15.70:0/2583761352 >> 172.21.15.70:0/2583761352 conn(0x7ff8c00a0060 msgr2=0x7ff8c00a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:05.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.058+0000 7ff8c5cd0640 1 -- 172.21.15.70:0/2583761352 shutdown_connections 2024-09-13T15:44:05.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.058+0000 7ff8c5cd0640 1 -- 172.21.15.70:0/2583761352 wait complete. 2024-09-13T15:44:05.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.059+0000 7f6a8163a640 1 -- 172.21.15.70:0/3230314440 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a7c06d2a0 msgr2=0x7f6a7c06d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.066 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.059+0000 7f6a8163a640 1 --2- 172.21.15.70:0/3230314440 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a7c06d2a0 0x7f6a7c06d680 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0097d0 tx=0x7f6a6c02a080 comp rx=0 tx=0).stop 2024-09-13T15:44:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.059+0000 7f6a8163a640 1 -- 172.21.15.70:0/3230314440 shutdown_connections 2024-09-13T15:44:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.059+0000 7f6a8163a640 1 --2- 172.21.15.70:0/3230314440 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6a7c12a640 0x7f6a7c12aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.059+0000 7f6a8163a640 1 --2- 172.21.15.70:0/3230314440 >> v2:172.21.15.70:3300/0 conn(0x7f6a7c06dbc0 0x7f6a7c12a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.059+0000 7f6a8163a640 1 --2- 172.21.15.70:0/3230314440 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a7c06d2a0 0x7f6a7c06d680 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.059+0000 7f6a8163a640 1 -- 172.21.15.70:0/3230314440 >> 172.21.15.70:0/3230314440 conn(0x7f6a7c09cde0 msgr2=0x7f6a7c09d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.059+0000 7f6a8163a640 1 -- 172.21.15.70:0/3230314440 shutdown_connections 2024-09-13T15:44:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.059+0000 7f6a8163a640 1 -- 172.21.15.70:0/3230314440 wait complete. 2024-09-13T15:44:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.059+0000 7ff8c5cd0640 1 Processor -- start 2024-09-13T15:44:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.060+0000 7ff8c5cd0640 1 -- start start 2024-09-13T15:44:05.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.060+0000 7f6a8163a640 1 Processor -- start 2024-09-13T15:44:05.068 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.060+0000 7f6a8163a640 1 -- start start 2024-09-13T15:44:05.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.060+0000 7ff8c5cd0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8c012a370 0x7ff8c012f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.060+0000 7ff8c5cd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8c012d330 0x7ff8c012f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.060+0000 7ff8c5cd0640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff8c012dd80 0x7ff8c012fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.060+0000 7ff8c5cd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c01316f0 con 0x7ff8c012d330 2024-09-13T15:44:05.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8c5cd0640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c009a120 con 0x7ff8c012a370 2024-09-13T15:44:05.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8c5cd0640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c0132990 con 0x7ff8c012dd80 2024-09-13T15:44:05.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8bf7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8c012a370 0x7ff8c012f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.069 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8bffff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff8c012dd80 0x7ff8c012fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8bffff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff8c012dd80 0x7ff8c012fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:39532/0 (socket says 172.21.15.70:39532) 2024-09-13T15:44:05.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8bf7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8c012a370 0x7ff8c012f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33050/0 (socket says 172.21.15.70:33050) 2024-09-13T15:44:05.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8bffff640 1 -- 172.21.15.70:0/1114447004 learned_addr learned my addr 172.21.15.70:0/1114447004 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:05.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8beffd640 1 --2- 172.21.15.70:0/1114447004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8c012d330 0x7ff8c012f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7f6a8163a640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6a7c06d2a0 0x7f6a7c1be250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.070 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7f6a8163a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a7c06dbc0 0x7f6a7c1be790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7f6a8163a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a7c12a640 0x7f6a7c08aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7f6a8163a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c08d000 con 0x7f6a7c12a640 2024-09-13T15:44:05.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7f6a8163a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c08d990 con 0x7f6a7c06dbc0 2024-09-13T15:44:05.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7f6a8163a640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c124ab0 con 0x7f6a7c06d2a0 2024-09-13T15:44:05.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7f6a7b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a7c12a640 0x7f6a7c08aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7f6a7affd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6a7c06d2a0 0x7f6a7c1be250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7f6a7affd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6a7c06d2a0 0x7f6a7c1be250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:39540/0 (socket says 172.21.15.70:39540) 2024-09-13T15:44:05.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7f6a7b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a7c12a640 0x7f6a7c08aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54002/0 (socket says 172.21.15.70:54002) 2024-09-13T15:44:05.071 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7f6a7affd640 1 -- 172.21.15.70:0/3959457541 learned_addr learned my addr 172.21.15.70:0/3959457541 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:05.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8beffd640 1 -- 172.21.15.70:0/1114447004 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff8c012dd80 msgr2=0x7ff8c012fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8beffd640 1 --2- 172.21.15.70:0/1114447004 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff8c012dd80 0x7ff8c012fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8beffd640 1 -- 172.21.15.70:0/1114447004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8c012a370 msgr2=0x7ff8c012f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8beffd640 1 --2- 172.21.15.70:0/1114447004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8c012a370 0x7ff8c012f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8beffd640 1 -- 172.21.15.70:0/1114447004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b40094d0 con 0x7ff8c012d330 2024-09-13T15:44:05.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8bf7fe640 1 --2- 172.21.15.70:0/1114447004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8c012a370 0x7ff8c012f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:05.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8bffff640 1 --2- 172.21.15.70:0/1114447004 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff8c012dd80 0x7ff8c012fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:05.072 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7ff8beffd640 1 --2- 172.21.15.70:0/1114447004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8c012d330 0x7ff8c012f800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4031000 tx=0x7ff8b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:05.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.061+0000 7f6a7a7fc640 1 --2- 172.21.15.70:0/3959457541 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a7c06dbc0 0x7f6a7c1be790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7f6a7b7fe640 1 -- 172.21.15.70:0/3959457541 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6a7c06d2a0 msgr2=0x7f6a7c1be250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7f6a7b7fe640 1 --2- 172.21.15.70:0/3959457541 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6a7c06d2a0 0x7f6a7c1be250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.073 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7f6a7b7fe640 1 -- 172.21.15.70:0/3959457541 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a7c06dbc0 msgr2=0x7f6a7c1be790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7f6a7b7fe640 1 --2- 172.21.15.70:0/3959457541 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a7c06dbc0 0x7f6a7c1be790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7f6a7b7fe640 1 -- 172.21.15.70:0/3959457541 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a700094d0 con 0x7f6a7c12a640 2024-09-13T15:44:05.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7f6a7affd640 1 --2- 172.21.15.70:0/3959457541 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6a7c06d2a0 0x7f6a7c1be250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:05.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7f6a7a7fc640 1 --2- 172.21.15.70:0/3959457541 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a7c06dbc0 0x7f6a7c1be790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:44:05.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7ff8a7fff640 1 -- 172.21.15.70:0/1114447004 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b403d850 con 0x7ff8c012d330 2024-09-13T15:44:05.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7ff8a7fff640 1 -- 172.21.15.70:0/1114447004 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b403d9b0 con 0x7ff8c012d330 2024-09-13T15:44:05.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7ff8a7fff640 1 -- 172.21.15.70:0/1114447004 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b403dd20 con 0x7ff8c012d330 2024-09-13T15:44:05.074 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7ff8c5cd0640 1 -- 172.21.15.70:0/1114447004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c0130cf0 con 0x7ff8c012d330 2024-09-13T15:44:05.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7f6a7b7fe640 1 --2- 172.21.15.70:0/3959457541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a7c12a640 0x7f6a7c08aff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009d90 tx=0x7f6a7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:05.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7ff8c5cd0640 1 -- 172.21.15.70:0/1114447004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c0132c50 con 0x7ff8c012d330 2024-09-13T15:44:05.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7f6a637fe640 1 -- 172.21.15.70:0/3959457541 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74014480 con 0x7f6a7c12a640 2024-09-13T15:44:05.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7f6a637fe640 1 -- 172.21.15.70:0/3959457541 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7400f040 con 0x7f6a7c12a640 2024-09-13T15:44:05.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7f6a637fe640 1 -- 172.21.15.70:0/3959457541 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74014840 con 0x7f6a7c12a640 2024-09-13T15:44:05.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7f6a8163a640 1 -- 172.21.15.70:0/3959457541 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6c009400 con 0x7f6a7c12a640 2024-09-13T15:44:05.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.062+0000 7f6a8163a640 1 -- 172.21.15.70:0/3959457541 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c1becd0 con 0x7f6a7c12a640 2024-09-13T15:44:05.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.063+0000 7ff8c5cd0640 1 -- 172.21.15.70:0/1114447004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c012d840 con 0x7ff8c012d330 2024-09-13T15:44:05.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.064+0000 7ff8a7fff640 1 -- 172.21.15.70:0/1114447004 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7ff8b4039040 con 0x7ff8c012d330 2024-09-13T15:44:05.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.064+0000 7f6a8163a640 1 -- 172.21.15.70:0/3959457541 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5c005250 con 0x7f6a7c12a640 2024-09-13T15:44:05.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.064+0000 7ff8a7fff640 1 -- 172.21.15.70:0/1114447004 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7ff8b407d070 con 0x7ff8c012d330 2024-09-13T15:44:05.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.064+0000 7f6a637fe640 1 -- 172.21.15.70:0/3959457541 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f6a740149a0 con 0x7f6a7c12a640 2024-09-13T15:44:05.077 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.065+0000 7f6a637fe640 1 -- 172.21.15.70:0/3959457541 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f6a7405d640 con 0x7f6a7c12a640 2024-09-13T15:44:05.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.071+0000 7ff8a7fff640 1 -- 172.21.15.70:0/1114447004 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7ff8b405e710 con 0x7ff8c012d330 2024-09-13T15:44:05.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.077+0000 7f6a637fe640 1 -- 172.21.15.70:0/3959457541 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f6a7403b2b0 con 0x7f6a7c12a640 2024-09-13T15:44:05.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.096+0000 7f1978253640 1 Processor -- start 2024-09-13T15:44:05.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.097+0000 7f1978253640 1 -- start start 2024-09-13T15:44:05.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.097+0000 7f1978253640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f197012df50 0x7f1970132650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.097+0000 7f1978253640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1970132b90 0x7f1970134f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.097+0000 7f1978253640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197006d5d0 con 0x7f197012d330 2024-09-13T15:44:05.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.097+0000 7f1978253640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19700971b0 con 0x7f197012df50 2024-09-13T15:44:05.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.097+0000 7f1978253640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19700979f0 con 0x7f1970132b90 2024-09-13T15:44:05.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.097+0000 7f19767f0640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1970132b90 0x7f1970134f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.097+0000 7f19757ee640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f197012df50 0x7f1970132650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.097+0000 7f19757ee640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f197012df50 0x7f1970132650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54008/0 (socket says 172.21.15.70:54008) 2024-09-13T15:44:05.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.097+0000 7f19767f0640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1970132b90 0x7f1970134f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:39542/0 (socket says 172.21.15.70:39542) 2024-09-13T15:44:05.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.097+0000 7f19757ee640 1 -- 172.21.15.70:0/1878879751 learned_addr learned my addr 172.21.15.70:0/1878879751 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:05.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.098+0000 7f19767f0640 1 -- 172.21.15.70:0/1878879751 >> v2:172.21.15.70:3300/0 conn(0x7f197012df50 msgr2=0x7f1970132650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:05.099 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.098+0000 7f19767f0640 1 --2- 172.21.15.70:0/1878879751 >> v2:172.21.15.70:3300/0 conn(0x7f197012df50 0x7f1970132650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.098+0000 7f19767f0640 1 -- 172.21.15.70:0/1878879751 >> v1:172.21.15.70:6789/0 conn(0x7f197012d330 legacy=0x7f197012d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:05.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.098+0000 7f19767f0640 1 -- 172.21.15.70:0/1878879751 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1970097620 con 0x7f1970132b90 2024-09-13T15:44:05.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.098+0000 7f19767f0640 1 --2- 172.21.15.70:0/1878879751 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1970132b90 0x7f1970134f80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1968009d90 tx=0x7f196802a4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8a951bc25c832a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:05.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.098+0000 7f1974fed640 1 -- 172.21.15.70:0/1878879751 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968007740 con 0x7f1970132b90 2024-09-13T15:44:05.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.098+0000 7f1974fed640 1 -- 172.21.15.70:0/1878879751 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196802d050 con 0x7f1970132b90 2024-09-13T15:44:05.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.098+0000 7f1974fed640 1 -- 172.21.15.70:0/1878879751 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968007ac0 con 0x7f1970132b90 2024-09-13T15:44:05.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.099+0000 7f1978253640 1 -- 172.21.15.70:0/1878879751 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1970132b90 msgr2=0x7f1970134f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.099+0000 7f1978253640 1 --2- 172.21.15.70:0/1878879751 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1970132b90 0x7f1970134f80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1968009d90 tx=0x7f196802a4e0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.099+0000 7f1978253640 1 -- 172.21.15.70:0/1878879751 shutdown_connections 2024-09-13T15:44:05.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.099+0000 7f1978253640 1 --2- 172.21.15.70:0/1878879751 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1970132b90 0x7f1970134f80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.099+0000 7f1978253640 1 --2- 172.21.15.70:0/1878879751 >> v2:172.21.15.70:3300/0 conn(0x7f197012df50 0x7f1970132650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.099+0000 7f1978253640 1 -- 172.21.15.70:0/1878879751 >> 172.21.15.70:0/1878879751 conn(0x7f19700a0060 msgr2=0x7f19700a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:05.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.099+0000 7f1978253640 1 -- 172.21.15.70:0/1878879751 shutdown_connections 2024-09-13T15:44:05.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.099+0000 7f1978253640 1 -- 172.21.15.70:0/1878879751 wait complete. 2024-09-13T15:44:05.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.100+0000 7f1978253640 1 Processor -- start 2024-09-13T15:44:05.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.100+0000 7f1978253640 1 -- start start 2024-09-13T15:44:05.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f1978253640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f197012d330 0x7f197012a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f1978253640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f197012df50 0x7f1970124d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f1978253640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1970132b90 0x7f1970125240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f1975fef640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f197012d330 0x7f197012a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f1975fef640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f197012d330 0x7f197012a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:39552/0 (socket says 172.21.15.70:39552) 2024-09-13T15:44:05.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f19757ee640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f197012df50 0x7f1970124d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f19757ee640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f197012df50 0x7f1970124d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33068/0 (socket says 172.21.15.70:33068) 2024-09-13T15:44:05.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f1978253640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19701266d0 con 0x7f1970132b90 2024-09-13T15:44:05.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f1978253640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19701272d0 con 0x7f197012df50 2024-09-13T15:44:05.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f1978253640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f197009a190 con 0x7f197012d330 2024-09-13T15:44:05.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f19767f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1970132b90 0x7f1970125240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f1975fef640 1 -- 172.21.15.70:0/2761661770 learned_addr learned my addr 172.21.15.70:0/2761661770 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:05.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7fd24b227640 1 Processor -- start 2024-09-13T15:44:05.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7fd24b227640 1 -- start start 2024-09-13T15:44:05.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f19757ee640 1 -- 172.21.15.70:0/2761661770 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f197012d330 msgr2=0x7f197012a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f19757ee640 1 --2- 172.21.15.70:0/2761661770 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f197012d330 0x7f197012a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f19757ee640 1 -- 172.21.15.70:0/2761661770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1970132b90 msgr2=0x7f1970125240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.101+0000 7f19757ee640 1 --2- 172.21.15.70:0/2761661770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1970132b90 0x7f1970125240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7f19757ee640 1 -- 172.21.15.70:0/2761661770 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1968009a70 con 0x7f197012df50 2024-09-13T15:44:05.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd24b227640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fd2441320b0 0x7fd244132490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd24b227640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd2441329d0 0x7fd24406be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd24b227640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd24406c370 0x7fd24406c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd24b227640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2440a0c60 con 0x7fd2441329d0 2024-09-13T15:44:05.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd24b227640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24406daf0 con 0x7fd24406c370 2024-09-13T15:44:05.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd24b227640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd244096f20 con 0x7fd2441320b0 2024-09-13T15:44:05.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd243fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd2441329d0 0x7fd24406be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd243fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd2441329d0 0x7fd24406be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54026/0 (socket says 172.21.15.70:54026) 2024-09-13T15:44:05.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd243fff640 1 -- 172.21.15.70:0/1651073032 learned_addr learned my addr 172.21.15.70:0/1651073032 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:05.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd2497c4640 1 --2- 172.21.15.70:0/1651073032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd24406c370 0x7fd24406c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd248fc3640 1 --2- 172.21.15.70:0/1651073032 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fd2441320b0 0x7fd244132490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7f19757ee640 1 --2- 172.21.15.70:0/2761661770 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f197012df50 0x7f1970124d00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f196000e850 tx=0x7f196000ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:05.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd2497c4640 1 -- 172.21.15.70:0/1651073032 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fd2441320b0 msgr2=0x7fd244132490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:05.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd2497c4640 1 --2- 172.21.15.70:0/1651073032 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fd2441320b0 0x7fd244132490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd2497c4640 1 -- 172.21.15.70:0/1651073032 >> v2:172.21.15.70:3300/0 conn(0x7fd2441329d0 msgr2=0x7fd24406be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:05.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd2497c4640 1 --2- 172.21.15.70:0/1651073032 >> v2:172.21.15.70:3300/0 conn(0x7fd2441329d0 0x7fd24406be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.102+0000 7fd2497c4640 1 -- 172.21.15.70:0/1651073032 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd244097940 con 0x7fd24406c370 2024-09-13T15:44:05.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.103+0000 7f19667fc640 1 -- 172.21.15.70:0/2761661770 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000c8e0 con 0x7f197012df50 2024-09-13T15:44:05.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.103+0000 7f19667fc640 1 -- 172.21.15.70:0/2761661770 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196000ca40 con 0x7f197012df50 2024-09-13T15:44:05.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.103+0000 7f19667fc640 1 -- 172.21.15.70:0/2761661770 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196000cdb0 con 0x7f197012df50 2024-09-13T15:44:05.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.103+0000 7f1978253640 1 -- 172.21.15.70:0/2761661770 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1970132420 con 0x7f197012df50 2024-09-13T15:44:05.109 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.103+0000 7fd2497c4640 1 --2- 172.21.15.70:0/1651073032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd24406c370 0x7fd24406c7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009d90 tx=0x7fd23c02a490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba53f0fad096e5df server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:05.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.103+0000 7f1978253640 1 -- 172.21.15.70:0/2761661770 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1970125780 con 0x7f197012df50 2024-09-13T15:44:05.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.104+0000 7fd2437fe640 1 -- 172.21.15.70:0/1651073032 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c007740 con 0x7fd24406c370 2024-09-13T15:44:05.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.104+0000 7fd2437fe640 1 -- 172.21.15.70:0/1651073032 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c02d050 con 0x7fd24406c370 2024-09-13T15:44:05.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.104+0000 7fd2437fe640 1 -- 172.21.15.70:0/1651073032 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c007ac0 con 0x7fd24406c370 2024-09-13T15:44:05.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.104+0000 7f19667fc640 1 -- 172.21.15.70:0/2761661770 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f1960013470 con 0x7f197012df50 2024-09-13T15:44:05.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.104+0000 7f1978253640 1 -- 172.21.15.70:0/2761661770 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1970127ac0 con 0x7f197012df50 2024-09-13T15:44:05.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.104+0000 7fd24b227640 1 -- 172.21.15.70:0/1651073032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd24406c370 msgr2=0x7fd24406c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.104+0000 7fd24b227640 1 --2- 172.21.15.70:0/1651073032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd24406c370 0x7fd24406c7a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009d90 tx=0x7fd23c02a490 comp rx=0 tx=0).stop 2024-09-13T15:44:05.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.105+0000 7fd24b227640 1 -- 172.21.15.70:0/1651073032 shutdown_connections 2024-09-13T15:44:05.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.105+0000 7fd24b227640 1 --2- 172.21.15.70:0/1651073032 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd24406c370 0x7fd24406c7a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.105+0000 7fd24b227640 1 --2- 172.21.15.70:0/1651073032 >> v2:172.21.15.70:3300/0 conn(0x7fd2441329d0 0x7fd24406be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.105+0000 7fd24b227640 1 --2- 172.21.15.70:0/1651073032 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fd2441320b0 0x7fd244132490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.105+0000 7fd24b227640 1 -- 172.21.15.70:0/1651073032 >> 172.21.15.70:0/1651073032 conn(0x7fd2440a00d0 msgr2=0x7fd2440a04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:05.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.105+0000 7fd24b227640 1 -- 172.21.15.70:0/1651073032 shutdown_connections 2024-09-13T15:44:05.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.105+0000 7fd24b227640 1 -- 172.21.15.70:0/1651073032 wait complete. 2024-09-13T15:44:05.114 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.104+0000 7f19667fc640 1 -- 172.21.15.70:0/2761661770 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f196005ca50 con 0x7f197012df50 2024-09-13T15:44:05.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.106+0000 7fd24b227640 1 Processor -- start 2024-09-13T15:44:05.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.106+0000 7fd24b227640 1 -- start start 2024-09-13T15:44:05.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.106+0000 7fd24b227640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd24406c370 0x7fd244126410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.106+0000 7fd24b227640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fd2441320b0 0x7fd244126950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.106+0000 7fd24b227640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2441329d0 0x7fd244126e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:05.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.106+0000 7fd24b227640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd244129820 con 0x7fd2441329d0 2024-09-13T15:44:05.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.106+0000 7fd24b227640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd244128800 con 0x7fd24406c370 2024-09-13T15:44:05.116 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.106+0000 7fd24b227640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd244129040 con 0x7fd2441320b0 2024-09-13T15:44:05.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.106+0000 7fd248fc3640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd24406c370 0x7fd244126410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.106+0000 7fd248fc3640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd24406c370 0x7fd244126410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33096/0 (socket says 172.21.15.70:33096) 2024-09-13T15:44:05.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.106+0000 7fd248fc3640 1 -- 172.21.15.70:0/3197648729 learned_addr learned my addr 172.21.15.70:0/3197648729 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:05.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.106+0000 7fd243fff640 1 --2- 172.21.15.70:0/3197648729 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fd2441320b0 0x7fd244126950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.107+0000 7fd2497c4640 1 --2- 172.21.15.70:0/3197648729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2441329d0 0x7fd244126e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:05.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.107+0000 7fd248fc3640 1 -- 172.21.15.70:0/3197648729 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fd2441320b0 msgr2=0x7fd244126950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.107+0000 7fd248fc3640 1 --2- 172.21.15.70:0/3197648729 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fd2441320b0 0x7fd244126950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.107+0000 7fd248fc3640 1 -- 172.21.15.70:0/3197648729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2441329d0 msgr2=0x7fd244126e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.107+0000 7fd248fc3640 1 --2- 172.21.15.70:0/3197648729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2441329d0 0x7fd244126e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.107+0000 7fd248fc3640 1 -- 172.21.15.70:0/3197648729 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c009a70 con 0x7fd24406c370 2024-09-13T15:44:05.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.107+0000 7fd2497c4640 1 --2- 172.21.15.70:0/3197648729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2441329d0 0x7fd244126e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:44:05.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.107+0000 7fd248fc3640 1 --2- 172.21.15.70:0/3197648729 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd24406c370 0x7fd244126410 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd23800e890 tx=0x7fd23800ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:05.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.111+0000 7fd2417fa640 1 -- 172.21.15.70:0/3197648729 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23800c940 con 0x7fd24406c370 2024-09-13T15:44:05.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.111+0000 7fd2417fa640 1 -- 172.21.15.70:0/3197648729 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23800caa0 con 0x7fd24406c370 2024-09-13T15:44:05.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.111+0000 7fd2417fa640 1 -- 172.21.15.70:0/3197648729 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23800ce30 con 0x7fd24406c370 2024-09-13T15:44:05.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.111+0000 7fd24b227640 1 -- 172.21.15.70:0/3197648729 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd244129bf0 con 0x7fd24406c370 2024-09-13T15:44:05.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.112+0000 7f19667fc640 1 -- 172.21.15.70:0/2761661770 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f196003a660 con 0x7f197012df50 2024-09-13T15:44:05.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.112+0000 7fd24b227640 1 -- 172.21.15.70:0/3197648729 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2441319c0 con 0x7fd24406c370 2024-09-13T15:44:05.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.113+0000 7fd24b227640 1 -- 172.21.15.70:0/3197648729 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd244131c70 con 0x7fd24406c370 2024-09-13T15:44:05.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.113+0000 7fd2417fa640 1 -- 172.21.15.70:0/3197648729 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fd238013470 con 0x7fd24406c370 2024-09-13T15:44:05.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.114+0000 7fd2417fa640 1 -- 172.21.15.70:0/3197648729 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fd23805c9d0 con 0x7fd24406c370 2024-09-13T15:44:05.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.120+0000 7fd2417fa640 1 -- 172.21.15.70:0/3197648729 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fd23803a6c0 con 0x7fd24406c370 2024-09-13T15:44:05.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.221+0000 7f6a8163a640 1 -- 172.21.15.70:0/3959457541 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6a5c0057d0 con 0x7f6a7c12a640 2024-09-13T15:44:05.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.221+0000 7f6a637fe640 1 -- 172.21.15.70:0/3959457541 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6a74018070 con 0x7f6a7c12a640 2024-09-13T15:44:05.223 INFO:teuthology.orchestra.run.smithi070.stdout:120259084290 2024-09-13T15:44:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.224+0000 7f6a8163a640 1 -- 172.21.15.70:0/3959457541 >> v1:172.21.15.70:6816/1229510400 conn(0x7f6a68047560 legacy=0x7f6a68049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.224+0000 7f6a8163a640 1 -- 172.21.15.70:0/3959457541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a7c12a640 msgr2=0x7f6a7c08aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.224+0000 7f6a8163a640 1 --2- 172.21.15.70:0/3959457541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a7c12a640 0x7f6a7c08aff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009d90 tx=0x7f6a7400c490 comp rx=0 tx=0).stop 2024-09-13T15:44:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.224+0000 7f6a8163a640 1 -- 172.21.15.70:0/3959457541 shutdown_connections 2024-09-13T15:44:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.224+0000 7f6a8163a640 1 --2- 172.21.15.70:0/3959457541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a7c12a640 0x7f6a7c08aff0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.224+0000 7f6a8163a640 1 --2- 172.21.15.70:0/3959457541 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6a7c06dbc0 0x7f6a7c1be790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.224+0000 7f6a8163a640 1 --2- 172.21.15.70:0/3959457541 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f6a7c06d2a0 0x7f6a7c1be250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.224+0000 7f6a8163a640 1 -- 172.21.15.70:0/3959457541 >> 172.21.15.70:0/3959457541 conn(0x7f6a7c09cde0 msgr2=0x7f6a7c123a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:05.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.224+0000 7f6a8163a640 1 -- 172.21.15.70:0/3959457541 shutdown_connections 2024-09-13T15:44:05.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.224+0000 7f6a8163a640 1 -- 172.21.15.70:0/3959457541 wait complete. 2024-09-13T15:44:05.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.226+0000 7ff8c5cd0640 1 -- 172.21.15.70:0/1114447004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff8c0131d40 con 0x7ff8c012d330 2024-09-13T15:44:05.227 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.226+0000 7ff8a7fff640 1 -- 172.21.15.70:0/1114447004 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8b4037020 con 0x7ff8c012d330 2024-09-13T15:44:05.227 INFO:teuthology.orchestra.run.smithi070.stdout:107374182403 2024-09-13T15:44:05.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.228+0000 7ff8c5cd0640 1 -- 172.21.15.70:0/1114447004 >> v1:172.21.15.70:6816/1229510400 conn(0x7ff8ac047560 legacy=0x7ff8ac049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.228+0000 7ff8c5cd0640 1 -- 172.21.15.70:0/1114447004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8c012d330 msgr2=0x7ff8c012f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.229 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.228+0000 7ff8c5cd0640 1 --2- 172.21.15.70:0/1114447004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8c012d330 0x7ff8c012f800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4031000 tx=0x7ff8b40046c0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.230 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.228+0000 7ff8c5cd0640 1 -- 172.21.15.70:0/1114447004 shutdown_connections 2024-09-13T15:44:05.230 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.228+0000 7ff8c5cd0640 1 --2- 172.21.15.70:0/1114447004 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff8c012dd80 0x7ff8c012fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.230 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.228+0000 7ff8c5cd0640 1 --2- 172.21.15.70:0/1114447004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8c012d330 0x7ff8c012f800 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.230 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.228+0000 7ff8c5cd0640 1 --2- 172.21.15.70:0/1114447004 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff8c012a370 0x7ff8c012f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.230 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.228+0000 7ff8c5cd0640 1 -- 172.21.15.70:0/1114447004 >> 172.21.15.70:0/1114447004 conn(0x7ff8c00a0060 msgr2=0x7ff8c009b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:05.230 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.228+0000 7ff8c5cd0640 1 -- 172.21.15.70:0/1114447004 shutdown_connections 2024-09-13T15:44:05.230 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.228+0000 7ff8c5cd0640 1 -- 172.21.15.70:0/1114447004 wait complete. 2024-09-13T15:44:05.241 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182404 got 107374182403 for osd.0 2024-09-13T15:44:05.241 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084291 got 120259084290 for osd.3 2024-09-13T15:44:05.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.256+0000 7f1978253640 1 -- 172.21.15.70:0/2761661770 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f197006a6b0 con 0x7f197012df50 2024-09-13T15:44:05.257 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.256+0000 7f19667fc640 1 -- 172.21.15.70:0/2761661770 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f196003dc80 con 0x7f197012df50 2024-09-13T15:44:05.257 INFO:teuthology.orchestra.run.smithi070.stdout:115964116994 2024-09-13T15:44:05.259 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.258+0000 7f1978253640 1 -- 172.21.15.70:0/2761661770 >> v1:172.21.15.70:6816/1229510400 conn(0x7f1954047560 legacy=0x7f1954049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.258+0000 7f1978253640 1 -- 172.21.15.70:0/2761661770 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f197012df50 msgr2=0x7f1970124d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.258+0000 7f1978253640 1 --2- 172.21.15.70:0/2761661770 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f197012df50 0x7f1970124d00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f196000e850 tx=0x7f196000ed10 comp rx=0 tx=0).stop 2024-09-13T15:44:05.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.258+0000 7f1978253640 1 -- 172.21.15.70:0/2761661770 shutdown_connections 2024-09-13T15:44:05.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.258+0000 7f1978253640 1 --2- 172.21.15.70:0/2761661770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1970132b90 0x7f1970125240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.258+0000 7f1978253640 1 --2- 172.21.15.70:0/2761661770 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f197012df50 0x7f1970124d00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.258+0000 7f1978253640 1 --2- 172.21.15.70:0/2761661770 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f197012d330 0x7f197012a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.258+0000 7f1978253640 1 -- 172.21.15.70:0/2761661770 >> 172.21.15.70:0/2761661770 conn(0x7f19700a0060 msgr2=0x7f197009b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:05.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.258+0000 7f1978253640 1 -- 172.21.15.70:0/2761661770 shutdown_connections 2024-09-13T15:44:05.260 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.258+0000 7f1978253640 1 -- 172.21.15.70:0/2761661770 wait complete. 2024-09-13T15:44:05.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964116995 got 115964116994 for osd.2 2024-09-13T15:44:05.271 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.270+0000 7fd24b227640 1 -- 172.21.15.70:0/3197648729 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd244132490 con 0x7fd24406c370 2024-09-13T15:44:05.272 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.271+0000 7fd2417fa640 1 -- 172.21.15.70:0/3197648729 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd238017030 con 0x7fd24406c370 2024-09-13T15:44:05.272 INFO:teuthology.orchestra.run.smithi070.stdout:111669149699 2024-09-13T15:44:05.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.273+0000 7fd24b227640 1 -- 172.21.15.70:0/3197648729 >> v1:172.21.15.70:6816/1229510400 conn(0x7fd2280475b0 legacy=0x7fd228049a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.275 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.273+0000 7fd24b227640 1 -- 172.21.15.70:0/3197648729 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd24406c370 msgr2=0x7fd244126410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.273+0000 7fd24b227640 1 --2- 172.21.15.70:0/3197648729 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd24406c370 0x7fd244126410 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd23800e890 tx=0x7fd23800ed50 comp rx=0 tx=0).stop 2024-09-13T15:44:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.273+0000 7fd24b227640 1 -- 172.21.15.70:0/3197648729 shutdown_connections 2024-09-13T15:44:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.273+0000 7fd24b227640 1 --2- 172.21.15.70:0/3197648729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2441329d0 0x7fd244126e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.273+0000 7fd24b227640 1 --2- 172.21.15.70:0/3197648729 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fd2441320b0 0x7fd244126950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.273+0000 7fd24b227640 1 --2- 172.21.15.70:0/3197648729 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd24406c370 0x7fd244126410 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.273+0000 7fd24b227640 1 -- 172.21.15.70:0/3197648729 >> 172.21.15.70:0/3197648729 conn(0x7fd2440a00d0 msgr2=0x7fd24409b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.273+0000 7fd24b227640 1 -- 172.21.15.70:0/3197648729 shutdown_connections 2024-09-13T15:44:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:05.273+0000 7fd24b227640 1 -- 172.21.15.70:0/3197648729 wait complete. 2024-09-13T15:44:05.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 111669149700 got 111669149699 for osd.1 2024-09-13T15:44:06.242 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T15:44:06.242 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T15:44:06.271 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T15:44:06.285 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T15:44:06.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.354+0000 7f8b97757640 1 Processor -- start 2024-09-13T15:44:06.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.355+0000 7f92f279d640 1 Processor -- start 2024-09-13T15:44:06.356 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.355+0000 7f92f279d640 1 -- start start 2024-09-13T15:44:06.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.355+0000 7f8b97757640 1 -- start start 2024-09-13T15:44:06.357 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.355+0000 7f92f279d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f92ec06c920 0x7f92ec06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.355+0000 7f92f279d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f92ec06d240 0x7f92ec12da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.355+0000 7f92f279d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92ec06d680 0x7f92ec12df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.355+0000 7f92f279d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ec071890 con 0x7f92ec06c920 2024-09-13T15:44:06.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.355+0000 7f92f279d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec096f10 con 0x7f92ec06d680 2024-09-13T15:44:06.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.355+0000 7f92f279d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec06bd00 con 0x7f92ec06d240 2024-09-13T15:44:06.358 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f92f0f9a640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f92ec06d240 0x7f92ec12da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.359 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.355+0000 7f8b97757640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f8b9012d330 0x7f8b9012d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.359 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f8b97757640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b9012a540 0x7f8b90135670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.359 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f92f1f9c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92ec06d680 0x7f92ec12df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.359 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f8b97757640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b9006d5d0 con 0x7f8b9012dd80 2024-09-13T15:44:06.360 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f92f0f9a640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f92ec06d240 0x7f92ec12da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:39584/0 (socket says 172.21.15.70:39584) 2024-09-13T15:44:06.360 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f8b97757640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b900971b0 con 0x7f8b9012a540 2024-09-13T15:44:06.360 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f8b97757640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b900979f0 con 0x7f8b9012d330 2024-09-13T15:44:06.360 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f92f0f9a640 1 -- 172.21.15.70:0/2792487833 learned_addr learned my addr 172.21.15.70:0/2792487833 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:06.360 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f92f179b640 1 --2- 172.21.15.70:0/2792487833 >> v2:172.21.15.70:3300/0 conn(0x7f92ec06c920 0x7f92ec06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f8b95cf4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b9012a540 0x7f8b90135670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f92f0f9a640 1 -- 172.21.15.70:0/2792487833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92ec06d680 msgr2=0x7f92ec12df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:06.361 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f8b95cf4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b9012a540 0x7f8b90135670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33114/0 (socket says 172.21.15.70:33114) 2024-09-13T15:44:06.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f8b95cf4640 1 -- 172.21.15.70:0/1506958759 learned_addr learned my addr 172.21.15.70:0/1506958759 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:06.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f8b954f3640 1 --2- 172.21.15.70:0/1506958759 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f8b9012d330 0x7f8b9012d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f92f0f9a640 1 --2- 172.21.15.70:0/2792487833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92ec06d680 0x7f92ec12df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f92f0f9a640 1 -- 172.21.15.70:0/2792487833 >> v2:172.21.15.70:3300/0 conn(0x7f92ec06c920 msgr2=0x7f92ec06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:06.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f92f0f9a640 1 --2- 172.21.15.70:0/2792487833 >> v2:172.21.15.70:3300/0 conn(0x7f92ec06c920 0x7f92ec06cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f92f0f9a640 1 -- 172.21.15.70:0/2792487833 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec06b600 con 0x7f92ec06d240 2024-09-13T15:44:06.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f92f1f9c640 1 --2- 172.21.15.70:0/2792487833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92ec06d680 0x7f92ec12df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:44:06.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f92f179b640 1 --2- 172.21.15.70:0/2792487833 >> v2:172.21.15.70:3300/0 conn(0x7f92ec06c920 0x7f92ec06cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:06.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f8b95cf4640 1 -- 172.21.15.70:0/1506958759 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f8b9012d330 msgr2=0x7f8b9012d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:06.363 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f8b95cf4640 1 --2- 172.21.15.70:0/1506958759 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f8b9012d330 0x7f8b9012d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f8b95cf4640 1 -- 172.21.15.70:0/1506958759 >> v1:172.21.15.70:6789/0 conn(0x7f8b9012dd80 legacy=0x7f8b90129e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:06.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.356+0000 7f8b95cf4640 1 -- 172.21.15.70:0/1506958759 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9012e1a0 con 0x7f8b9012a540 2024-09-13T15:44:06.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.357+0000 7f92f0f9a640 1 --2- 172.21.15.70:0/2792487833 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f92ec06d240 0x7f92ec12da40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009870 tx=0x7f92dc02a090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f21d9d15274e39 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:06.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.357+0000 7f92e3fff640 1 -- 172.21.15.70:0/2792487833 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc02ade0 con 0x7f92ec06d240 2024-09-13T15:44:06.364 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.357+0000 7f92e3fff640 1 -- 172.21.15.70:0/2792487833 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc032440 con 0x7f92ec06d240 2024-09-13T15:44:06.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.357+0000 7f92e3fff640 1 -- 172.21.15.70:0/2792487833 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0327d0 con 0x7f92ec06d240 2024-09-13T15:44:06.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.357+0000 7f8b95cf4640 1 --2- 172.21.15.70:0/1506958759 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b9012a540 0x7f8b90135670 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009e80 tx=0x7f8b8802a590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f63bd5afcbcabee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:06.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.357+0000 7f8b87fff640 1 -- 172.21.15.70:0/1506958759 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88031600 con 0x7f8b9012a540 2024-09-13T15:44:06.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.357+0000 7f8b87fff640 1 -- 172.21.15.70:0/1506958759 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8802d050 con 0x7f8b9012a540 2024-09-13T15:44:06.365 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.358+0000 7f8b87fff640 1 -- 172.21.15.70:0/1506958759 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88031980 con 0x7f8b9012a540 2024-09-13T15:44:06.366 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.358+0000 7f92f279d640 1 -- 172.21.15.70:0/2792487833 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f92ec06d240 msgr2=0x7f92ec12da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.366 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.358+0000 7f92f279d640 1 --2- 172.21.15.70:0/2792487833 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f92ec06d240 0x7f92ec12da40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009870 tx=0x7f92dc02a090 comp rx=0 tx=0).stop 2024-09-13T15:44:06.366 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.358+0000 7f92f279d640 1 -- 172.21.15.70:0/2792487833 shutdown_connections 2024-09-13T15:44:06.366 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.358+0000 7f92f279d640 1 --2- 172.21.15.70:0/2792487833 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92ec06d680 0x7f92ec12df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.367 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.358+0000 7f92f279d640 1 --2- 172.21.15.70:0/2792487833 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f92ec06d240 0x7f92ec12da40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.367 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.358+0000 7f92f279d640 1 --2- 172.21.15.70:0/2792487833 >> v2:172.21.15.70:3300/0 conn(0x7f92ec06c920 0x7f92ec06cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.367 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.358+0000 7f92f279d640 1 -- 172.21.15.70:0/2792487833 >> 172.21.15.70:0/2792487833 conn(0x7f92ec09c8c0 msgr2=0x7f92ec09ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:06.367 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.358+0000 7f92f279d640 1 -- 172.21.15.70:0/2792487833 shutdown_connections 2024-09-13T15:44:06.367 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.358+0000 7f92f279d640 1 -- 172.21.15.70:0/2792487833 wait complete. 2024-09-13T15:44:06.367 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.358+0000 7f8b97757640 1 -- 172.21.15.70:0/1506958759 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b9012a540 msgr2=0x7f8b90135670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.358+0000 7f8b97757640 1 --2- 172.21.15.70:0/1506958759 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b9012a540 0x7f8b90135670 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009e80 tx=0x7f8b8802a590 comp rx=0 tx=0).stop 2024-09-13T15:44:06.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.359+0000 7f8b97757640 1 -- 172.21.15.70:0/1506958759 shutdown_connections 2024-09-13T15:44:06.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.359+0000 7f8b97757640 1 --2- 172.21.15.70:0/1506958759 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b9012a540 0x7f8b90135670 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.359+0000 7f8b97757640 1 --2- 172.21.15.70:0/1506958759 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f8b9012d330 0x7f8b9012d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.359+0000 7f8b97757640 1 -- 172.21.15.70:0/1506958759 >> 172.21.15.70:0/1506958759 conn(0x7f8b900a0060 msgr2=0x7f8b900a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:06.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.359+0000 7f8b97757640 1 -- 172.21.15.70:0/1506958759 shutdown_connections 2024-09-13T15:44:06.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.359+0000 7f8b97757640 1 -- 172.21.15.70:0/1506958759 wait complete. 2024-09-13T15:44:06.368 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f92f279d640 1 Processor -- start 2024-09-13T15:44:06.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f92f279d640 1 -- start start 2024-09-13T15:44:06.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f8b97757640 1 Processor -- start 2024-09-13T15:44:06.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f8b97757640 1 -- start start 2024-09-13T15:44:06.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f92f279d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f92ec06c920 0x7f92ec093b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f92f279d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92ec06d240 0x7f92ec094060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f92f279d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92ec06d680 0x7f92ec1c2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f92f0f9a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92ec06d240 0x7f92ec094060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f92f0f9a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92ec06d240 0x7f92ec094060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33130/0 (socket says 172.21.15.70:33130) 2024-09-13T15:44:06.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f92f1f9c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92ec06d680 0x7f92ec1c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f92f1f9c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92ec06d680 0x7f92ec1c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54052/0 (socket says 172.21.15.70:54052) 2024-09-13T15:44:06.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f92f0f9a640 1 -- 172.21.15.70:0/210241937 learned_addr learned my addr 172.21.15.70:0/210241937 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:06.370 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f92f179b640 1 --2- 172.21.15.70:0/210241937 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f92ec06c920 0x7f92ec093b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f92f279d640 1 -- 172.21.15.70:0/210241937 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec1220e0 con 0x7f92ec06d680 2024-09-13T15:44:06.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f92f279d640 1 -- 172.21.15.70:0/210241937 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec0952a0 con 0x7f92ec06d240 2024-09-13T15:44:06.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.360+0000 7f92f279d640 1 -- 172.21.15.70:0/210241937 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec1223f0 con 0x7f92ec06c920 2024-09-13T15:44:06.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f8b97757640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b9012a540 0x7f8b9012ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f8b97757640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f8b9012d330 0x7f8b9012f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.371 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f8b97757640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b9012dd80 0x7f8b9012f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f92f1f9c640 1 -- 172.21.15.70:0/210241937 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f92ec06c920 msgr2=0x7f92ec093b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f92f1f9c640 1 --2- 172.21.15.70:0/210241937 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f92ec06c920 0x7f92ec093b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f92f1f9c640 1 -- 172.21.15.70:0/210241937 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92ec06d240 msgr2=0x7f92ec094060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f92f1f9c640 1 --2- 172.21.15.70:0/210241937 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92ec06d240 0x7f92ec094060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f92f1f9c640 1 -- 172.21.15.70:0/210241937 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e8009400 con 0x7f92ec06d680 2024-09-13T15:44:06.372 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f92f0f9a640 1 --2- 172.21.15.70:0/210241937 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92ec06d240 0x7f92ec094060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:06.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f8b97757640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9009a190 con 0x7f8b9012a540 2024-09-13T15:44:06.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f8b97757640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b90131640 con 0x7f8b9012dd80 2024-09-13T15:44:06.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f8b97757640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b90131270 con 0x7f8b9012d330 2024-09-13T15:44:06.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f8b95cf4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b9012dd80 0x7f8b9012f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.373 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f92f1f9c640 1 --2- 172.21.15.70:0/210241937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92ec06d680 0x7f92ec1c2110 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f92ec06c450 tx=0x7f92e400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:06.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f8b95cf4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b9012dd80 0x7f8b9012f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33140/0 (socket says 172.21.15.70:33140) 2024-09-13T15:44:06.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f8b95cf4640 1 -- 172.21.15.70:0/3996394970 learned_addr learned my addr 172.21.15.70:0/3996394970 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:06.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f8b94cf2640 1 --2- 172.21.15.70:0/3996394970 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f8b9012d330 0x7f8b9012f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.374 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f8b954f3640 1 --2- 172.21.15.70:0/3996394970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b9012a540 0x7f8b9012ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.361+0000 7f92e1ffb640 1 -- 172.21.15.70:0/210241937 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4015400 con 0x7f92ec06d680 2024-09-13T15:44:06.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.362+0000 7f8b95cf4640 1 -- 172.21.15.70:0/3996394970 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f8b9012d330 msgr2=0x7f8b9012f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.362+0000 7f8b95cf4640 1 --2- 172.21.15.70:0/3996394970 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f8b9012d330 0x7f8b9012f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.362+0000 7f8b95cf4640 1 -- 172.21.15.70:0/3996394970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b9012a540 msgr2=0x7f8b9012ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.375 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.362+0000 7f92e1ffb640 1 -- 172.21.15.70:0/210241937 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e400f040 con 0x7f92ec06d680 2024-09-13T15:44:06.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.362+0000 7f92e1ffb640 1 -- 172.21.15.70:0/210241937 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e40157c0 con 0x7f92ec06d680 2024-09-13T15:44:06.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.362+0000 7f8b95cf4640 1 --2- 172.21.15.70:0/3996394970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b9012a540 0x7f8b9012ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.362+0000 7f8b95cf4640 1 -- 172.21.15.70:0/3996394970 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b88009b30 con 0x7f8b9012dd80 2024-09-13T15:44:06.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.362+0000 7f92f279d640 1 -- 172.21.15.70:0/210241937 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92dc0094d0 con 0x7f92ec06d680 2024-09-13T15:44:06.376 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.362+0000 7f92f279d640 1 -- 172.21.15.70:0/210241937 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ec0945a0 con 0x7f92ec06d680 2024-09-13T15:44:06.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.363+0000 7f8b95cf4640 1 --2- 172.21.15.70:0/3996394970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b9012dd80 0x7f8b9012f9b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8b88034040 tx=0x7f8b88004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:06.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.363+0000 7f8b85ffb640 1 -- 172.21.15.70:0/3996394970 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8803d860 con 0x7f8b9012dd80 2024-09-13T15:44:06.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.363+0000 7f8b85ffb640 1 -- 172.21.15.70:0/3996394970 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8803d9e0 con 0x7f8b9012dd80 2024-09-13T15:44:06.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.363+0000 7f8b97757640 1 -- 172.21.15.70:0/3996394970 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b90131bd0 con 0x7f8b9012dd80 2024-09-13T15:44:06.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.363+0000 7f8b85ffb640 1 -- 172.21.15.70:0/3996394970 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8803dd90 con 0x7f8b9012dd80 2024-09-13T15:44:06.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.363+0000 7f92e1ffb640 1 -- 172.21.15.70:0/210241937 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f92e401e3b0 con 0x7f92ec06d680 2024-09-13T15:44:06.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.363+0000 7f92f279d640 1 -- 172.21.15.70:0/210241937 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ec12e470 con 0x7f92ec06d680 2024-09-13T15:44:06.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.364+0000 7f8b97757640 1 -- 172.21.15.70:0/3996394970 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b901300d0 con 0x7f8b9012dd80 2024-09-13T15:44:06.378 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.364+0000 7f8b85ffb640 1 -- 172.21.15.70:0/3996394970 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f8b8803b400 con 0x7f8b9012dd80 2024-09-13T15:44:06.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.364+0000 7f92e1ffb640 1 -- 172.21.15.70:0/210241937 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f92e405cbb0 con 0x7f92ec06d680 2024-09-13T15:44:06.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.365+0000 7f8b85ffb640 1 -- 172.21.15.70:0/3996394970 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f8b88081450 con 0x7f8b9012dd80 2024-09-13T15:44:06.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.365+0000 7f8b97757640 1 -- 172.21.15.70:0/3996394970 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b90126410 con 0x7f8b9012dd80 2024-09-13T15:44:06.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.372+0000 7f92e1ffb640 1 -- 172.21.15.70:0/210241937 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f92e403a7c0 con 0x7f92ec06d680 2024-09-13T15:44:06.379 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.372+0000 7f8b85ffb640 1 -- 172.21.15.70:0/3996394970 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f8b88041080 con 0x7f8b9012dd80 2024-09-13T15:44:06.407 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.406+0000 7f21e6ffb640 1 Processor -- start 2024-09-13T15:44:06.408 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.406+0000 7f21e6ffb640 1 -- start start 2024-09-13T15:44:06.408 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.406+0000 7f9cd2ee5640 1 Processor -- start 2024-09-13T15:44:06.408 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.406+0000 7f9cd2ee5640 1 -- start start 2024-09-13T15:44:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f21e6ffb640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f21e012d780 0x7f21e012db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f21e6ffb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f21e006c8b0 0x7f21e006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f21e6ffb640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e012d330 con 0x7f21e006d210 2024-09-13T15:44:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f21e6ffb640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e006dc70 con 0x7f21e006c8b0 2024-09-13T15:44:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f21e4d97640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f21e012d780 0x7f21e012db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f21e6ffb640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21e009d4c0 con 0x7f21e012d780 2024-09-13T15:44:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f21dffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f21e006c8b0 0x7f21e006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f21e4d97640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f21e012d780 0x7f21e012db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:39614/0 (socket says 172.21.15.70:39614) 2024-09-13T15:44:06.411 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f21e4d97640 1 -- 172.21.15.70:0/1446015958 learned_addr learned my addr 172.21.15.70:0/1446015958 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:06.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f9cd2ee5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9ccc06cdd0 0x7f9ccc06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f9cd2ee5640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ccc06d6f0 0x7f9ccc12a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f9cd2ee5640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9ccc12a640 0x7f9ccc12aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f9cd2ee5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ccc06e930 con 0x7f9ccc06cdd0 2024-09-13T15:44:06.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f9cd2ee5640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc06daf0 con 0x7f9ccc06d6f0 2024-09-13T15:44:06.412 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f9cd2ee5640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc06c070 con 0x7f9ccc12a640 2024-09-13T15:44:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f9ccbfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ccc06d6f0 0x7f9ccc12a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f9ccbfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ccc06d6f0 0x7f9ccc12a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33152/0 (socket says 172.21.15.70:33152) 2024-09-13T15:44:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f9ccbfff640 1 -- 172.21.15.70:0/794202311 learned_addr learned my addr 172.21.15.70:0/794202311 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f9cd0c81640 1 --2- 172.21.15.70:0/794202311 >> v2:172.21.15.70:3300/0 conn(0x7f9ccc06cdd0 0x7f9ccc06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f9cd1482640 1 --2- 172.21.15.70:0/794202311 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9ccc12a640 0x7f9ccc12aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f21df7fe640 1 -- 172.21.15.70:0/1446015958 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630071323 0 0) 0x7f21e012d330 con 0x7f21e006d210 2024-09-13T15:44:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.407+0000 7f21df7fe640 1 -- 172.21.15.70:0/1446015958 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21d0003520 con 0x7f21e006d210 2024-09-13T15:44:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.408+0000 7f9ccbfff640 1 -- 172.21.15.70:0/794202311 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9ccc12a640 msgr2=0x7f9ccc12aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.408+0000 7f9ccbfff640 1 --2- 172.21.15.70:0/794202311 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9ccc12a640 0x7f9ccc12aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.408+0000 7f9ccbfff640 1 -- 172.21.15.70:0/794202311 >> v2:172.21.15.70:3300/0 conn(0x7f9ccc06cdd0 msgr2=0x7f9ccc06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.408+0000 7f9ccbfff640 1 --2- 172.21.15.70:0/794202311 >> v2:172.21.15.70:3300/0 conn(0x7f9ccc06cdd0 0x7f9ccc06d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.408+0000 7f9ccbfff640 1 -- 172.21.15.70:0/794202311 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc06f350 con 0x7f9ccc06d6f0 2024-09-13T15:44:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.408+0000 7f21dffff640 1 -- 172.21.15.70:0/1446015958 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f21e012d780 msgr2=0x7f21e012db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:06.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.408+0000 7f21dffff640 1 --2- 172.21.15.70:0/1446015958 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f21e012d780 0x7f21e012db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.408+0000 7f21dffff640 1 -- 172.21.15.70:0/1446015958 >> v1:172.21.15.70:6789/0 conn(0x7f21e006d210 legacy=0x7f21e0134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.408+0000 7f21dffff640 1 -- 172.21.15.70:0/1446015958 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e0067710 con 0x7f21e006c8b0 2024-09-13T15:44:06.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.408+0000 7f21e4d97640 1 --2- 172.21.15.70:0/1446015958 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f21e012d780 0x7f21e012db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T15:44:06.415 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.409+0000 7f9ccbfff640 1 --2- 172.21.15.70:0/794202311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ccc06d6f0 0x7f9ccc12a100 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80098a0 tx=0x7f9cb802a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95c3882d59370ad1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:06.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.409+0000 7f21dffff640 1 --2- 172.21.15.70:0/1446015958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e006c8b0 0x7f21e006ccd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0097a0 tx=0x7f21cc02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f91f0d04e6aafce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:06.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.409+0000 7f9ccb7fe640 1 -- 172.21.15.70:0/794202311 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb802ade0 con 0x7f9ccc06d6f0 2024-09-13T15:44:06.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.409+0000 7f9ccb7fe640 1 -- 172.21.15.70:0/794202311 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb8032440 con 0x7f9ccc06d6f0 2024-09-13T15:44:06.416 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.409+0000 7f9ccb7fe640 1 -- 172.21.15.70:0/794202311 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80327d0 con 0x7f9ccc06d6f0 2024-09-13T15:44:06.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.409+0000 7f21df7fe640 1 -- 172.21.15.70:0/1446015958 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc02adb0 con 0x7f21e006c8b0 2024-09-13T15:44:06.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.409+0000 7f21df7fe640 1 -- 172.21.15.70:0/1446015958 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc032440 con 0x7f21e006c8b0 2024-09-13T15:44:06.417 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.409+0000 7f21df7fe640 1 -- 172.21.15.70:0/1446015958 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0327b0 con 0x7f21e006c8b0 2024-09-13T15:44:06.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f21e6ffb640 1 -- 172.21.15.70:0/1446015958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e006c8b0 msgr2=0x7f21e006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f21e6ffb640 1 --2- 172.21.15.70:0/1446015958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e006c8b0 0x7f21e006ccd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0097a0 tx=0x7f21cc02a080 comp rx=0 tx=0).stop 2024-09-13T15:44:06.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f21e6ffb640 1 -- 172.21.15.70:0/1446015958 shutdown_connections 2024-09-13T15:44:06.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f21e6ffb640 1 --2- 172.21.15.70:0/1446015958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e006c8b0 0x7f21e006ccd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f21e6ffb640 1 --2- 172.21.15.70:0/1446015958 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f21e012d780 0x7f21e012db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f21e6ffb640 1 -- 172.21.15.70:0/1446015958 >> 172.21.15.70:0/1446015958 conn(0x7f21e00a02e0 msgr2=0x7f21e009c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:06.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f21e6ffb640 1 -- 172.21.15.70:0/1446015958 shutdown_connections 2024-09-13T15:44:06.418 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f21e6ffb640 1 -- 172.21.15.70:0/1446015958 wait complete. 2024-09-13T15:44:06.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/794202311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ccc06d6f0 msgr2=0x7f9ccc12a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f9cd2ee5640 1 --2- 172.21.15.70:0/794202311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ccc06d6f0 0x7f9ccc12a100 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80098a0 tx=0x7f9cb802a0d0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/794202311 shutdown_connections 2024-09-13T15:44:06.419 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f9cd2ee5640 1 --2- 172.21.15.70:0/794202311 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9ccc12a640 0x7f9ccc12aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f9cd2ee5640 1 --2- 172.21.15.70:0/794202311 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ccc06d6f0 0x7f9ccc12a100 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80098a0 tx=0x7f9cb802a0d0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f9cd2ee5640 1 --2- 172.21.15.70:0/794202311 >> v2:172.21.15.70:3300/0 conn(0x7f9ccc06cdd0 0x7f9ccc06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/794202311 >> 172.21.15.70:0/794202311 conn(0x7f9ccc0973c0 msgr2=0x7f9ccc0977d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:06.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.410+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/794202311 shutdown_connections 2024-09-13T15:44:06.420 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.411+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/794202311 wait complete. 2024-09-13T15:44:06.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.411+0000 7f21e6ffb640 1 Processor -- start 2024-09-13T15:44:06.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.411+0000 7f21e6ffb640 1 -- start start 2024-09-13T15:44:06.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.411+0000 7f21e6ffb640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21e006c8b0 0x7f21e008c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.421 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.411+0000 7f21e6ffb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e006d210 0x7f21e008c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.411+0000 7f21e6ffb640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f21e012d780 0x7f21e01c2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.411+0000 7f21dffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e006d210 0x7f21e008c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21dffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e006d210 0x7f21e008c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54096/0 (socket says 172.21.15.70:54096) 2024-09-13T15:44:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21e6ffb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e008d320 con 0x7f21e006d210 2024-09-13T15:44:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21e6ffb640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e008ee70 con 0x7f21e006c8b0 2024-09-13T15:44:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21e6ffb640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21e008f410 con 0x7f21e012d780 2024-09-13T15:44:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21e4d97640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21e006c8b0 0x7f21e008c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21dffff640 1 -- 172.21.15.70:0/2240473429 learned_addr learned my addr 172.21.15.70:0/2240473429 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21e5598640 1 --2- 172.21.15.70:0/2240473429 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f21e012d780 0x7f21e01c2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21dffff640 1 -- 172.21.15.70:0/2240473429 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f21e012d780 msgr2=0x7f21e01c2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21dffff640 1 --2- 172.21.15.70:0/2240473429 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f21e012d780 0x7f21e01c2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21dffff640 1 -- 172.21.15.70:0/2240473429 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21e006c8b0 msgr2=0x7f21e008c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21dffff640 1 --2- 172.21.15.70:0/2240473429 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21e006c8b0 0x7f21e008c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21dffff640 1 -- 172.21.15.70:0/2240473429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc009400 con 0x7f21e006d210 2024-09-13T15:44:06.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21e5598640 1 --2- 172.21.15.70:0/2240473429 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f21e012d780 0x7f21e01c2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T15:44:06.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21e4d97640 1 --2- 172.21.15.70:0/2240473429 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21e006c8b0 0x7f21e008c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:06.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f9cd2ee5640 1 Processor -- start 2024-09-13T15:44:06.423 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f9cd2ee5640 1 -- start start 2024-09-13T15:44:06.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21dffff640 1 --2- 172.21.15.70:0/2240473429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e006d210 0x7f21e008c700 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f21cc02a0c0 tx=0x7f21cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:06.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21dd7fa640 1 -- 172.21.15.70:0/2240473429 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc003aa0 con 0x7f21e006d210 2024-09-13T15:44:06.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21dd7fa640 1 -- 172.21.15.70:0/2240473429 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc003c00 con 0x7f21e006d210 2024-09-13T15:44:06.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21dd7fa640 1 -- 172.21.15.70:0/2240473429 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0325b0 con 0x7f21e006d210 2024-09-13T15:44:06.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.412+0000 7f21e6ffb640 1 -- 172.21.15.70:0/2240473429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e008f6b0 con 0x7f21e006d210 2024-09-13T15:44:06.424 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.413+0000 7f21e6ffb640 1 -- 172.21.15.70:0/2240473429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e008e300 con 0x7f21e006d210 2024-09-13T15:44:06.425 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.413+0000 7f9cd2ee5640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9ccc06cdd0 0x7f9ccc12cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.413+0000 7f9cd2ee5640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ccc12a640 0x7f9ccc12b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.413+0000 7f9cd2ee5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ccc12b730 0x7f9ccc12bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f9ccbfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ccc12a640 0x7f9ccc12b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f9ccbfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ccc12a640 0x7f9ccc12b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33176/0 (socket says 172.21.15.70:33176) 2024-09-13T15:44:06.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f9ccbfff640 1 -- 172.21.15.70:0/624374610 learned_addr learned my addr 172.21.15.70:0/624374610 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:06.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f9cd0c81640 1 --2- 172.21.15.70:0/624374610 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9ccc06cdd0 0x7f9ccc12cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f9cd2ee5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc124a40 con 0x7f9ccc12b730 2024-09-13T15:44:06.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/624374610 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc12e330 con 0x7f9ccc12a640 2024-09-13T15:44:06.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f9cd1482640 1 --2- 172.21.15.70:0/624374610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ccc12b730 0x7f9ccc12bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.426 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f21dd7fa640 1 -- 172.21.15.70:0/2240473429 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f21cc032710 con 0x7f21e006d210 2024-09-13T15:44:06.427 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/624374610 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc12c630 con 0x7f9ccc06cdd0 2024-09-13T15:44:06.427 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f21e6ffb640 1 -- 172.21.15.70:0/2240473429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e0067400 con 0x7f21e006d210 2024-09-13T15:44:06.427 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f21dd7fa640 1 -- 172.21.15.70:0/2240473429 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f21cc07d510 con 0x7f21e006d210 2024-09-13T15:44:06.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f9cd1482640 1 -- 172.21.15.70:0/624374610 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9ccc06cdd0 msgr2=0x7f9ccc12cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f9cd1482640 1 --2- 172.21.15.70:0/624374610 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9ccc06cdd0 0x7f9ccc12cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f9cd1482640 1 -- 172.21.15.70:0/624374610 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ccc12a640 msgr2=0x7f9ccc12b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f9cd1482640 1 --2- 172.21.15.70:0/624374610 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ccc12a640 0x7f9ccc12b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.414+0000 7f9cd1482640 1 -- 172.21.15.70:0/624374610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb80094d0 con 0x7f9ccc12b730 2024-09-13T15:44:06.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.415+0000 7f9cd1482640 1 --2- 172.21.15.70:0/624374610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ccc12b730 0x7f9ccc12bb90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400efc0 tx=0x7f9cc400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:06.428 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.421+0000 7f21dd7fa640 1 -- 172.21.15.70:0/2240473429 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f21cc006660 con 0x7f21e006d210 2024-09-13T15:44:06.429 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.421+0000 7f9cc97fa640 1 -- 172.21.15.70:0/624374610 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4009e90 con 0x7f9ccc12b730 2024-09-13T15:44:06.429 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.421+0000 7f9cc97fa640 1 -- 172.21.15.70:0/624374610 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc400f040 con 0x7f9ccc12b730 2024-09-13T15:44:06.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.421+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/624374610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ccc12d930 con 0x7f9ccc12b730 2024-09-13T15:44:06.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.422+0000 7f9cc97fa640 1 -- 172.21.15.70:0/624374610 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40146a0 con 0x7f9ccc12b730 2024-09-13T15:44:06.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.422+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/624374610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ccc1c29c0 con 0x7f9ccc12b730 2024-09-13T15:44:06.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.423+0000 7f9cc97fa640 1 -- 172.21.15.70:0/624374610 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f9cc4014800 con 0x7f9ccc12b730 2024-09-13T15:44:06.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.423+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/624374610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ccc127820 con 0x7f9ccc12b730 2024-09-13T15:44:06.430 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.423+0000 7f9cc97fa640 1 -- 172.21.15.70:0/624374610 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f9cc405d610 con 0x7f9ccc12b730 2024-09-13T15:44:06.431 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.430+0000 7f9cc97fa640 1 -- 172.21.15.70:0/624374610 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f9ccc127820 con 0x7f9ccc12b730 2024-09-13T15:44:06.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.537+0000 7f92f279d640 1 -- 172.21.15.70:0/210241937 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f92ec12e7a0 con 0x7f92ec06d680 2024-09-13T15:44:06.538 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.537+0000 7f92e1ffb640 1 -- 172.21.15.70:0/210241937 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f92e401a070 con 0x7f92ec06d680 2024-09-13T15:44:06.538 INFO:teuthology.orchestra.run.smithi070.stdout:120259084292 2024-09-13T15:44:06.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.539+0000 7f92f279d640 1 -- 172.21.15.70:0/210241937 >> v1:172.21.15.70:6816/1229510400 conn(0x7f92d0047560 legacy=0x7f92d0049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.539+0000 7f92f279d640 1 -- 172.21.15.70:0/210241937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92ec06d680 msgr2=0x7f92ec1c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.539+0000 7f92f279d640 1 --2- 172.21.15.70:0/210241937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92ec06d680 0x7f92ec1c2110 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f92ec06c450 tx=0x7f92e400d4a0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.539+0000 7f92f279d640 1 -- 172.21.15.70:0/210241937 shutdown_connections 2024-09-13T15:44:06.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.539+0000 7f92f279d640 1 --2- 172.21.15.70:0/210241937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92ec06d680 0x7f92ec1c2110 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.539+0000 7f92f279d640 1 --2- 172.21.15.70:0/210241937 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f92ec06d240 0x7f92ec094060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.539+0000 7f92f279d640 1 --2- 172.21.15.70:0/210241937 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f92ec06c920 0x7f92ec093b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.539+0000 7f92f279d640 1 -- 172.21.15.70:0/210241937 >> 172.21.15.70:0/210241937 conn(0x7f92ec09c8c0 msgr2=0x7f92ec1b5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:06.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.540+0000 7f92f279d640 1 -- 172.21.15.70:0/210241937 shutdown_connections 2024-09-13T15:44:06.541 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.540+0000 7f92f279d640 1 -- 172.21.15.70:0/210241937 wait complete. 2024-09-13T15:44:06.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.547+0000 7f8b97757640 1 -- 172.21.15.70:0/3996394970 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8b9006a6b0 con 0x7f8b9012dd80 2024-09-13T15:44:06.548 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.547+0000 7f8b85ffb640 1 -- 172.21.15.70:0/3996394970 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8b88036070 con 0x7f8b9012dd80 2024-09-13T15:44:06.548 INFO:teuthology.orchestra.run.smithi070.stdout:107374182404 2024-09-13T15:44:06.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.550+0000 7f8b97757640 1 -- 172.21.15.70:0/3996394970 >> v1:172.21.15.70:6816/1229510400 conn(0x7f8b74047560 legacy=0x7f8b74049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.551 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.550+0000 7f8b97757640 1 -- 172.21.15.70:0/3996394970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b9012dd80 msgr2=0x7f8b9012f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.550+0000 7f8b97757640 1 --2- 172.21.15.70:0/3996394970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b9012dd80 0x7f8b9012f9b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8b88034040 tx=0x7f8b88004a40 comp rx=0 tx=0).stop 2024-09-13T15:44:06.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.550+0000 7f8b97757640 1 -- 172.21.15.70:0/3996394970 shutdown_connections 2024-09-13T15:44:06.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.550+0000 7f8b97757640 1 --2- 172.21.15.70:0/3996394970 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b9012dd80 0x7f8b9012f9b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.550+0000 7f8b97757640 1 --2- 172.21.15.70:0/3996394970 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f8b9012d330 0x7f8b9012f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.550+0000 7f8b97757640 1 --2- 172.21.15.70:0/3996394970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b9012a540 0x7f8b9012ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.550+0000 7f8b97757640 1 -- 172.21.15.70:0/3996394970 >> 172.21.15.70:0/3996394970 conn(0x7f8b900a0060 msgr2=0x7f8b900984d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:06.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.550+0000 7f8b97757640 1 -- 172.21.15.70:0/3996394970 shutdown_connections 2024-09-13T15:44:06.552 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.550+0000 7f8b97757640 1 -- 172.21.15.70:0/3996394970 wait complete. 2024-09-13T15:44:06.556 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084291 got 120259084292 for osd.3 2024-09-13T15:44:06.556 DEBUG:teuthology.parallel:result is None 2024-09-13T15:44:06.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 107374182404 got 107374182404 for osd.0 2024-09-13T15:44:06.564 DEBUG:teuthology.parallel:result is None 2024-09-13T15:44:06.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.576+0000 7f21e6ffb640 1 -- 172.21.15.70:0/2240473429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f21e0099300 con 0x7f21e006d210 2024-09-13T15:44:06.577 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.576+0000 7f21dd7fa640 1 -- 172.21.15.70:0/2240473429 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f21cc037090 con 0x7f21e006d210 2024-09-13T15:44:06.578 INFO:teuthology.orchestra.run.smithi070.stdout:115964116996 2024-09-13T15:44:06.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.578+0000 7f21e6ffb640 1 -- 172.21.15.70:0/2240473429 >> v1:172.21.15.70:6816/1229510400 conn(0x7f21c4047560 legacy=0x7f21c4049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.578+0000 7f21e6ffb640 1 -- 172.21.15.70:0/2240473429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e006d210 msgr2=0x7f21e008c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.578+0000 7f21e6ffb640 1 --2- 172.21.15.70:0/2240473429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e006d210 0x7f21e008c700 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f21cc02a0c0 tx=0x7f21cc0046c0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.578+0000 7f21e6ffb640 1 -- 172.21.15.70:0/2240473429 shutdown_connections 2024-09-13T15:44:06.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.578+0000 7f21e6ffb640 1 --2- 172.21.15.70:0/2240473429 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f21e012d780 0x7f21e01c2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.578+0000 7f21e6ffb640 1 --2- 172.21.15.70:0/2240473429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21e006d210 0x7f21e008c700 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.578+0000 7f21e6ffb640 1 --2- 172.21.15.70:0/2240473429 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f21e006c8b0 0x7f21e008c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.578+0000 7f21e6ffb640 1 -- 172.21.15.70:0/2240473429 >> 172.21.15.70:0/2240473429 conn(0x7f21e00a02e0 msgr2=0x7f21e0124d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:06.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.579+0000 7f21e6ffb640 1 -- 172.21.15.70:0/2240473429 shutdown_connections 2024-09-13T15:44:06.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.579+0000 7f21e6ffb640 1 -- 172.21.15.70:0/2240473429 wait complete. 2024-09-13T15:44:06.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 115964116995 got 115964116996 for osd.2 2024-09-13T15:44:06.591 DEBUG:teuthology.parallel:result is None 2024-09-13T15:44:06.593 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.592+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/624374610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9ccc127820 con 0x7f9ccc12b730 2024-09-13T15:44:06.594 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.592+0000 7f9cc97fa640 1 -- 172.21.15.70:0/624374610 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ccc127820 con 0x7f9ccc12b730 2024-09-13T15:44:06.594 INFO:teuthology.orchestra.run.smithi070.stdout:111669149700 2024-09-13T15:44:06.595 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.594+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/624374610 >> v1:172.21.15.70:6816/1229510400 conn(0x7f9cb00475b0 legacy=0x7f9cb0049a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.594+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/624374610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ccc12b730 msgr2=0x7f9ccc12bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.594+0000 7f9cd2ee5640 1 --2- 172.21.15.70:0/624374610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ccc12b730 0x7f9ccc12bb90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400efc0 tx=0x7f9cc400c490 comp rx=0 tx=0).stop 2024-09-13T15:44:06.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.594+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/624374610 shutdown_connections 2024-09-13T15:44:06.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.594+0000 7f9cd2ee5640 1 --2- 172.21.15.70:0/624374610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ccc12b730 0x7f9ccc12bb90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.594+0000 7f9cd2ee5640 1 --2- 172.21.15.70:0/624374610 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ccc12a640 0x7f9ccc12b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.594+0000 7f9cd2ee5640 1 --2- 172.21.15.70:0/624374610 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9ccc06cdd0 0x7f9ccc12cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.594+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/624374610 >> 172.21.15.70:0/624374610 conn(0x7f9ccc0973c0 msgr2=0x7f9ccc070480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:06.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.594+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/624374610 shutdown_connections 2024-09-13T15:44:06.596 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.594+0000 7f9cd2ee5640 1 -- 172.21.15.70:0/624374610 wait complete. 2024-09-13T15:44:06.606 INFO:tasks.ceph.ceph_manager.ceph:need seq 111669149700 got 111669149700 for osd.1 2024-09-13T15:44:06.606 DEBUG:teuthology.parallel:result is None 2024-09-13T15:44:06.606 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T15:44:06.606 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T15:44:06.743 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.741+0000 7ff5aa8e9640 1 Processor -- start 2024-09-13T15:44:06.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.741+0000 7ff5aa8e9640 1 -- start start 2024-09-13T15:44:06.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff5aa8e9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5a406c600 0x7ff5a4132710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff5aa8e9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5a4132c50 0x7ff5a4135040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff5aa8e9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a4097940 con 0x7ff5a406bb10 2024-09-13T15:44:06.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff5aa8e9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a4097500 con 0x7ff5a4132c50 2024-09-13T15:44:06.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff5aa8e9640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a406d100 con 0x7ff5a406c600 2024-09-13T15:44:06.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff5a8e86640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5a4132c50 0x7ff5a4135040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff5a8e86640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5a4132c50 0x7ff5a4135040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54100/0 (socket says 172.21.15.70:54100) 2024-09-13T15:44:06.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff5a3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff5a406bb10 0x7ff5a406bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55442/0 (socket says 172.21.15.70:55442) 2024-09-13T15:44:06.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff5a8e86640 1 -- 172.21.15.70:0/607081047 learned_addr learned my addr 172.21.15.70:0/607081047 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:06.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff59bfff640 1 --2- 172.21.15.70:0/607081047 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5a406c600 0x7ff5a4132710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff5a8e86640 1 -- 172.21.15.70:0/607081047 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5a406c600 msgr2=0x7ff5a4132710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:06.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff5a8e86640 1 --2- 172.21.15.70:0/607081047 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5a406c600 0x7ff5a4132710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff5a8e86640 1 -- 172.21.15.70:0/607081047 >> v1:172.21.15.70:6789/0 conn(0x7ff5a406bb10 legacy=0x7ff5a406bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff5a8e86640 1 -- 172.21.15.70:0/607081047 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a406da40 con 0x7ff5a4132c50 2024-09-13T15:44:06.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff5a37fe640 1 -- 172.21.15.70:0/607081047 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557204923 0 0) 0x7ff5a4097940 con 0x7ff5a406bb10 2024-09-13T15:44:06.745 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.742+0000 7ff59bfff640 1 --2- 172.21.15.70:0/607081047 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5a406c600 0x7ff5a4132710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:06.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.743+0000 7ff5a8e86640 1 --2- 172.21.15.70:0/607081047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a4132c50 0x7ff5a4135040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009e50 tx=0x7ff59c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38e880596d18b95 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:06.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.743+0000 7ff5a37fe640 1 -- 172.21.15.70:0/607081047 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c031600 con 0x7ff5a4132c50 2024-09-13T15:44:06.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.743+0000 7ff5a37fe640 1 -- 172.21.15.70:0/607081047 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c02d050 con 0x7ff5a4132c50 2024-09-13T15:44:06.748 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.743+0000 7ff5a37fe640 1 -- 172.21.15.70:0/607081047 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c031980 con 0x7ff5a4132c50 2024-09-13T15:44:06.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.744+0000 7ff5aa8e9640 1 -- 172.21.15.70:0/607081047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a4132c50 msgr2=0x7ff5a4135040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.744+0000 7ff5aa8e9640 1 --2- 172.21.15.70:0/607081047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a4132c50 0x7ff5a4135040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff59c009e50 tx=0x7ff59c02a610 comp rx=0 tx=0).stop 2024-09-13T15:44:06.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.744+0000 7ff5aa8e9640 1 -- 172.21.15.70:0/607081047 shutdown_connections 2024-09-13T15:44:06.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.744+0000 7ff5aa8e9640 1 --2- 172.21.15.70:0/607081047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a4132c50 0x7ff5a4135040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.744+0000 7ff5aa8e9640 1 --2- 172.21.15.70:0/607081047 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5a406c600 0x7ff5a4132710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.744+0000 7ff5aa8e9640 1 -- 172.21.15.70:0/607081047 >> 172.21.15.70:0/607081047 conn(0x7ff5a409fe20 msgr2=0x7ff5a40a0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:06.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.744+0000 7ff5aa8e9640 1 -- 172.21.15.70:0/607081047 shutdown_connections 2024-09-13T15:44:06.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.744+0000 7ff5aa8e9640 1 -- 172.21.15.70:0/607081047 wait complete. 2024-09-13T15:44:06.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5aa8e9640 1 Processor -- start 2024-09-13T15:44:06.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5aa8e9640 1 -- start start 2024-09-13T15:44:06.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5aa8e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a406bb10 0x7ff5a4128360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5aa8e9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5a406c600 0x7ff5a41288a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.749 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5aa8e9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5a4132c50 0x7ff5a4128de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5aa8e9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a412ee50 con 0x7ff5a406bb10 2024-09-13T15:44:06.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5aa8e9640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a412f130 con 0x7ff5a406c600 2024-09-13T15:44:06.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5aa8e9640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a4129f40 con 0x7ff5a4132c50 2024-09-13T15:44:06.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5a3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a406bb10 0x7ff5a4128360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5a3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a406bb10 0x7ff5a4128360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54112/0 (socket says 172.21.15.70:54112) 2024-09-13T15:44:06.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5a3fff640 1 -- 172.21.15.70:0/3400113393 learned_addr learned my addr 172.21.15.70:0/3400113393 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:06.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5a8e86640 1 --2- 172.21.15.70:0/3400113393 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5a4132c50 0x7ff5a4128de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff59bfff640 1 --2- 172.21.15.70:0/3400113393 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5a406c600 0x7ff5a41288a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5a3fff640 1 -- 172.21.15.70:0/3400113393 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5a4132c50 msgr2=0x7ff5a4128de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5a3fff640 1 --2- 172.21.15.70:0/3400113393 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5a4132c50 0x7ff5a4128de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5a3fff640 1 -- 172.21.15.70:0/3400113393 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5a406c600 msgr2=0x7ff5a41288a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5a3fff640 1 --2- 172.21.15.70:0/3400113393 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5a406c600 0x7ff5a41288a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.750 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5a3fff640 1 -- 172.21.15.70:0/3400113393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c009400 con 0x7ff5a406bb10 2024-09-13T15:44:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff59bfff640 1 --2- 172.21.15.70:0/3400113393 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5a406c600 0x7ff5a41288a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.745+0000 7ff5a3fff640 1 --2- 172.21.15.70:0/3400113393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a406bb10 0x7ff5a4128360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff59400c4a0 tx=0x7ff59400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.746+0000 7ff5a17fa640 1 -- 172.21.15.70:0/3400113393 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5940074e0 con 0x7ff5a406bb10 2024-09-13T15:44:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.746+0000 7ff5a17fa640 1 -- 172.21.15.70:0/3400113393 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff594007640 con 0x7ff5a406bb10 2024-09-13T15:44:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.746+0000 7ff5aa8e9640 1 -- 172.21.15.70:0/3400113393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59c009b30 con 0x7ff5a406bb10 2024-09-13T15:44:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.746+0000 7ff5a17fa640 1 -- 172.21.15.70:0/3400113393 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff594004a00 con 0x7ff5a406bb10 2024-09-13T15:44:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.746+0000 7ff5aa8e9640 1 -- 172.21.15.70:0/3400113393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a412a940 con 0x7ff5a406bb10 2024-09-13T15:44:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.747+0000 7ff5aa8e9640 1 -- 172.21.15.70:0/3400113393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a412e830 con 0x7ff5a406bb10 2024-09-13T15:44:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.747+0000 7ff5a17fa640 1 -- 172.21.15.70:0/3400113393 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7ff594004b60 con 0x7ff5a406bb10 2024-09-13T15:44:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.747+0000 7ff5a17fa640 1 -- 172.21.15.70:0/3400113393 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7ff59405bdd0 con 0x7ff5a406bb10 2024-09-13T15:44:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.751+0000 7ff5a17fa640 1 -- 172.21.15.70:0/3400113393 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7ff5a412e830 con 0x7ff5a406bb10 2024-09-13T15:44:06.882 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.881+0000 7ff5aa8e9640 1 -- 172.21.15.70:0/3400113393 --> v1:172.21.15.70:6816/1229510400 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5a406bef0 con 0x7ff584047560 2024-09-13T15:44:06.883 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.882+0000 7ff5a17fa640 1 -- 172.21.15.70:0/3400113393 <== mgr.24114 v1:172.21.15.70:6816/1229510400 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34750 (unknown 2965378022 0 1846488942) 0x7ff5a406bef0 con 0x7ff584047560 2024-09-13T15:44:06.883 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:44:06.884 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-13T15:44:06.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.884+0000 7ff5aa8e9640 1 -- 172.21.15.70:0/3400113393 >> v1:172.21.15.70:6816/1229510400 conn(0x7ff584047560 legacy=0x7ff584049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.884+0000 7ff5aa8e9640 1 -- 172.21.15.70:0/3400113393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a406bb10 msgr2=0x7ff5a4128360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.884+0000 7ff5aa8e9640 1 --2- 172.21.15.70:0/3400113393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a406bb10 0x7ff5a4128360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff59400c4a0 tx=0x7ff59400c960 comp rx=0 tx=0).stop 2024-09-13T15:44:06.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.884+0000 7ff5aa8e9640 1 -- 172.21.15.70:0/3400113393 shutdown_connections 2024-09-13T15:44:06.885 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.884+0000 7ff5aa8e9640 1 --2- 172.21.15.70:0/3400113393 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7ff5a4132c50 0x7ff5a4128de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.884+0000 7ff5aa8e9640 1 --2- 172.21.15.70:0/3400113393 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff5a406c600 0x7ff5a41288a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.884+0000 7ff5aa8e9640 1 --2- 172.21.15.70:0/3400113393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a406bb10 0x7ff5a4128360 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.884+0000 7ff5aa8e9640 1 -- 172.21.15.70:0/3400113393 >> 172.21.15.70:0/3400113393 conn(0x7ff5a409fe20 msgr2=0x7ff5a409b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:06.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.884+0000 7ff5aa8e9640 1 -- 172.21.15.70:0/3400113393 shutdown_connections 2024-09-13T15:44:06.886 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.884+0000 7ff5aa8e9640 1 -- 172.21.15.70:0/3400113393 wait complete. 2024-09-13T15:44:06.896 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-13T15:44:05.540582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":128992,"kb_used_data":2208,"kb_used_omap":17,"kb_used_meta":126702,"kb_avail":377358368,"statfs":{"total":386547056640,"available":386414968832,"internally_reserved":0,"allocated":2260992,"data_stored":1554908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":129743093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001114"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599825+0000","last_change":"2024-09-13T15:43:56.791803+0000","last_active":"2024-09-13T15:44:01.599825+0000","last_peered":"2024-09-13T15:44:01.599825+0000","last_clean":"2024-09-13T15:44:01.599825+0000","last_became_active":"2024-09-13T15:43:56.791454+0000","last_became_peered":"2024-09-13T15:43:56.791454+0000","last_unstale":"2024-09-13T15:44:01.599825+0000","last_undegraded":"2024-09-13T15:44:01.599825+0000","last_fullsized":"2024-09-13T15:44:01.599825+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:38.078282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035985100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599965+0000","last_change":"2024-09-13T15:43:54.574295+0000","last_active":"2024-09-13T15:44:01.599965+0000","last_peered":"2024-09-13T15:44:01.599965+0000","last_clean":"2024-09-13T15:44:01.599965+0000","last_became_active":"2024-09-13T15:43:54.573978+0000","last_became_peered":"2024-09-13T15:43:54.573978+0000","last_unstale":"2024-09-13T15:44:01.599965+0000","last_undegraded":"2024-09-13T15:44:01.599965+0000","last_fullsized":"2024-09-13T15:44:01.599965+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:09.974326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054553999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599736+0000","last_change":"2024-09-13T15:43:56.792140+0000","last_active":"2024-09-13T15:44:01.599736+0000","last_peered":"2024-09-13T15:44:01.599736+0000","last_clean":"2024-09-13T15:44:01.599736+0000","last_became_active":"2024-09-13T15:43:56.791789+0000","last_became_peered":"2024-09-13T15:43:56.791789+0000","last_unstale":"2024-09-13T15:44:01.599736+0000","last_undegraded":"2024-09-13T15:44:01.599736+0000","last_fullsized":"2024-09-13T15:44:01.599736+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:51.347592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011766770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599970+0000","last_change":"2024-09-13T15:43:54.573858+0000","last_active":"2024-09-13T15:44:01.599970+0000","last_peered":"2024-09-13T15:44:01.599970+0000","last_clean":"2024-09-13T15:44:01.599970+0000","last_became_active":"2024-09-13T15:43:54.573540+0000","last_became_peered":"2024-09-13T15:43:54.573540+0000","last_unstale":"2024-09-13T15:44:01.599970+0000","last_undegraded":"2024-09-13T15:44:01.599970+0000","last_fullsized":"2024-09-13T15:44:01.599970+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:50.925568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024452499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599730+0000","last_change":"2024-09-13T15:43:56.584865+0000","last_active":"2024-09-13T15:44:01.599730+0000","last_peered":"2024-09-13T15:44:01.599730+0000","last_clean":"2024-09-13T15:44:01.599730+0000","last_became_active":"2024-09-13T15:43:56.584517+0000","last_became_peered":"2024-09-13T15:43:56.584517+0000","last_unstale":"2024-09-13T15:44:01.599730+0000","last_undegraded":"2024-09-13T15:44:01.599730+0000","last_fullsized":"2024-09-13T15:44:01.599730+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:29:26.245037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094542300000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T15:43:56.583705+0000","last_change":"2024-09-13T15:43:56.582866+0000","last_active":"2024-09-13T15:43:56.583705+0000","last_peered":"2024-09-13T15:43:56.583705+0000","last_clean":"2024-09-13T15:43:56.583705+0000","last_became_active":"2024-09-13T15:43:56.582550+0000","last_became_peered":"2024-09-13T15:43:56.582550+0000","last_unstale":"2024-09-13T15:43:56.583705+0000","last_undegraded":"2024-09-13T15:43:56.583705+0000","last_fullsized":"2024-09-13T15:43:56.583705+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:26.074766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089033199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599643+0000","last_change":"2024-09-13T15:43:56.585264+0000","last_active":"2024-09-13T15:44:01.599643+0000","last_peered":"2024-09-13T15:44:01.599643+0000","last_clean":"2024-09-13T15:44:01.599643+0000","last_became_active":"2024-09-13T15:43:56.584633+0000","last_became_peered":"2024-09-13T15:43:56.584633+0000","last_unstale":"2024-09-13T15:44:01.599643+0000","last_undegraded":"2024-09-13T15:44:01.599643+0000","last_fullsized":"2024-09-13T15:44:01.599643+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:29.252314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034895899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599737+0000","last_change":"2024-09-13T15:43:56.583583+0000","last_active":"2024-09-13T15:44:01.599737+0000","last_peered":"2024-09-13T15:44:01.599737+0000","last_clean":"2024-09-13T15:44:01.599737+0000","last_became_active":"2024-09-13T15:43:56.582549+0000","last_became_peered":"2024-09-13T15:43:56.582549+0000","last_unstale":"2024-09-13T15:44:01.599737+0000","last_undegraded":"2024-09-13T15:44:01.599737+0000","last_fullsized":"2024-09-13T15:44:01.599737+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:58:03.411133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276560000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'63","reported_seq":140,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599581+0000","last_change":"2024-09-13T15:43:54.574397+0000","last_active":"2024-09-13T15:44:01.599581+0000","last_peered":"2024-09-13T15:44:01.599581+0000","last_clean":"2024-09-13T15:44:01.599581+0000","last_became_active":"2024-09-13T15:43:54.574044+0000","last_became_peered":"2024-09-13T15:43:54.574044+0000","last_unstale":"2024-09-13T15:44:01.599581+0000","last_undegraded":"2024-09-13T15:44:01.599581+0000","last_fullsized":"2024-09-13T15:44:01.599581+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_clean_scrub_stamp":"2024-09-13T15:42:21.689345+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:48:10.619565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":28,"seq":120259084292,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":268,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":274432,"data_stored":95335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.215}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]}]},{"osd":2,"up_from":27,"seq":115964116996,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":256,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":262144,"data_stored":91751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]}]},{"osd":1,"up_from":26,"seq":111669149700,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":868352,"data_stored":685703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":32435774},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]}]},{"osd":0,"up_from":25,"seq":107374182404,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":836,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":856064,"data_stored":682119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T15:44:06.897 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T15:44:06.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.992+0000 7f1fa76b7640 1 Processor -- start 2024-09-13T15:44:06.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.992+0000 7f1fa76b7640 1 -- start start 2024-09-13T15:44:06.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa76b7640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1fa00953e0 0x7f1fa00957c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa76b7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1fa006ddf0 0x7f1fa006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa76b7640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1fa006cfc0 0x7f1fa006d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:06.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa76b7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa006e3a0 con 0x7f1fa006ddf0 2024-09-13T15:44:06.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa76b7640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa0097d40 con 0x7f1fa00953e0 2024-09-13T15:44:06.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa76b7640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa006f110 con 0x7f1fa006cfc0 2024-09-13T15:44:06.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa4c52640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1fa006ddf0 0x7f1fa006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa5453640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1fa00953e0 0x7f1fa00957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa5c54640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1fa006cfc0 0x7f1fa006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:06.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa5453640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1fa00953e0 0x7f1fa00957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33204/0 (socket says 172.21.15.70:33204) 2024-09-13T15:44:06.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa5453640 1 -- 172.21.15.70:0/2910083750 learned_addr learned my addr 172.21.15.70:0/2910083750 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:06.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa4c52640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1fa006ddf0 0x7f1fa006ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54126/0 (socket says 172.21.15.70:54126) 2024-09-13T15:44:06.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa5c54640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1fa006cfc0 0x7f1fa006d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:39672/0 (socket says 172.21.15.70:39672) 2024-09-13T15:44:06.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa4c52640 1 -- 172.21.15.70:0/2910083750 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1fa006cfc0 msgr2=0x7f1fa006d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:06.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa4c52640 1 --2- 172.21.15.70:0/2910083750 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1fa006cfc0 0x7f1fa006d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa4c52640 1 -- 172.21.15.70:0/2910083750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1fa00953e0 msgr2=0x7f1fa00957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:06.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa4c52640 1 --2- 172.21.15.70:0/2910083750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1fa00953e0 0x7f1fa00957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa4c52640 1 -- 172.21.15.70:0/2910083750 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa006eb70 con 0x7f1fa006ddf0 2024-09-13T15:44:06.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa5453640 1 --2- 172.21.15.70:0/2910083750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1fa00953e0 0x7f1fa00957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:06.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.993+0000 7f1fa5c54640 1 --2- 172.21.15.70:0/2910083750 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1fa006cfc0 0x7f1fa006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:06.997 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.994+0000 7f1fa4c52640 1 --2- 172.21.15.70:0/2910083750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1fa006ddf0 0x7f1fa006ca80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1f900097a0 tx=0x7f1f90029fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219aea53fcb15e0d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:06.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.994+0000 7f1f97fff640 1 -- 172.21.15.70:0/2910083750 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9002ad20 con 0x7f1fa006ddf0 2024-09-13T15:44:06.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.994+0000 7f1f97fff640 1 -- 172.21.15.70:0/2910083750 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9002ae80 con 0x7f1fa006ddf0 2024-09-13T15:44:06.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.994+0000 7f1f97fff640 1 -- 172.21.15.70:0/2910083750 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f90032660 con 0x7f1fa006ddf0 2024-09-13T15:44:06.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.994+0000 7f1fa76b7640 1 -- 172.21.15.70:0/2910083750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1fa006ddf0 msgr2=0x7f1fa006ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:06.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.994+0000 7f1fa76b7640 1 --2- 172.21.15.70:0/2910083750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1fa006ddf0 0x7f1fa006ca80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1f900097a0 tx=0x7f1f90029fd0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.994+0000 7f1fa76b7640 1 -- 172.21.15.70:0/2910083750 shutdown_connections 2024-09-13T15:44:06.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.994+0000 7f1fa76b7640 1 --2- 172.21.15.70:0/2910083750 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1fa006cfc0 0x7f1fa006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.994+0000 7f1fa76b7640 1 --2- 172.21.15.70:0/2910083750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1fa006ddf0 0x7f1fa006ca80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.994+0000 7f1fa76b7640 1 --2- 172.21.15.70:0/2910083750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1fa00953e0 0x7f1fa00957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:06.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.994+0000 7f1fa76b7640 1 -- 172.21.15.70:0/2910083750 >> 172.21.15.70:0/2910083750 conn(0x7f1fa0098b10 msgr2=0x7f1fa00963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:06.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.995+0000 7f1fa76b7640 1 -- 172.21.15.70:0/2910083750 shutdown_connections 2024-09-13T15:44:06.998 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.995+0000 7f1fa76b7640 1 -- 172.21.15.70:0/2910083750 wait complete. 2024-09-13T15:44:07.000 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.995+0000 7f1fa76b7640 1 Processor -- start 2024-09-13T15:44:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.995+0000 7f1fa76b7640 1 -- start start 2024-09-13T15:44:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa76b7640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1fa006cfc0 0x7f1fa0122af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa76b7640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1fa006ddf0 0x7f1fa0123030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa76b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1fa00953e0 0x7f1fa0123570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa76b7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa0124a20 con 0x7f1fa00953e0 2024-09-13T15:44:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa76b7640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa0124590 con 0x7f1fa006ddf0 2024-09-13T15:44:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa76b7640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa008bc70 con 0x7f1fa006cfc0 2024-09-13T15:44:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa5c54640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1fa00953e0 0x7f1fa0123570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa5453640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1fa006cfc0 0x7f1fa0122af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa4c52640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1fa006ddf0 0x7f1fa0123030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa5c54640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1fa00953e0 0x7f1fa0123570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54130/0 (socket says 172.21.15.70:54130) 2024-09-13T15:44:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa5c54640 1 -- 172.21.15.70:0/47394722 learned_addr learned my addr 172.21.15.70:0/47394722 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa5c54640 1 -- 172.21.15.70:0/47394722 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1fa006cfc0 msgr2=0x7f1fa0122af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa5c54640 1 --2- 172.21.15.70:0/47394722 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1fa006cfc0 0x7f1fa0122af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa5c54640 1 -- 172.21.15.70:0/47394722 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1fa006ddf0 msgr2=0x7f1fa0123030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa5c54640 1 --2- 172.21.15.70:0/47394722 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1fa006ddf0 0x7f1fa0123030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa5c54640 1 -- 172.21.15.70:0/47394722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9800bc80 con 0x7f1fa00953e0 2024-09-13T15:44:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa5453640 1 --2- 172.21.15.70:0/47394722 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1fa006cfc0 0x7f1fa0122af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa5c54640 1 --2- 172.21.15.70:0/47394722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1fa00953e0 0x7f1fa0123570 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800e390 tx=0x7f1f9800e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1f95ffb640 1 -- 172.21.15.70:0/47394722 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98010050 con 0x7f1fa00953e0 2024-09-13T15:44:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1f95ffb640 1 -- 172.21.15.70:0/47394722 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f98011040 con 0x7f1fa00953e0 2024-09-13T15:44:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1fa76b7640 1 -- 172.21.15.70:0/47394722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9c0094d0 con 0x7f1fa00953e0 2024-09-13T15:44:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.996+0000 7f1f95ffb640 1 -- 172.21.15.70:0/47394722 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98009740 con 0x7f1fa00953e0 2024-09-13T15:44:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.997+0000 7f1fa76b7640 1 -- 172.21.15.70:0/47394722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f90009400 con 0x7f1fa00953e0 2024-09-13T15:44:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.997+0000 7f1f95ffb640 1 -- 172.21.15.70:0/47394722 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f1f980098a0 con 0x7f1fa00953e0 2024-09-13T15:44:07.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.998+0000 7f1fa76b7640 1 -- 172.21.15.70:0/47394722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa0095f10 con 0x7f1fa00953e0 2024-09-13T15:44:07.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:06.998+0000 7f1f95ffb640 1 -- 172.21.15.70:0/47394722 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f1f9805ef20 con 0x7f1fa00953e0 2024-09-13T15:44:07.003 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.001+0000 7f1f95ffb640 1 -- 172.21.15.70:0/47394722 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f1f9803cb90 con 0x7f1fa00953e0 2024-09-13T15:44:07.137 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.135+0000 7f1fa76b7640 1 -- 172.21.15.70:0/47394722 --> v1:172.21.15.70:6816/1229510400 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1fa008b470 con 0x7f1f84047590 2024-09-13T15:44:07.138 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.137+0000 7f1f95ffb640 1 -- 172.21.15.70:0/47394722 <== mgr.24114 v1:172.21.15.70:6816/1229510400 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34750 (unknown 2965378022 0 1846488942) 0x7f1fa008b470 con 0x7f1f84047590 2024-09-13T15:44:07.138 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:44:07.139 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-13T15:44:07.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.138+0000 7f1fa76b7640 1 -- 172.21.15.70:0/47394722 >> v1:172.21.15.70:6816/1229510400 conn(0x7f1f84047590 legacy=0x7f1f84049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.138+0000 7f1fa76b7640 1 -- 172.21.15.70:0/47394722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1fa00953e0 msgr2=0x7f1fa0123570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.138+0000 7f1fa76b7640 1 --2- 172.21.15.70:0/47394722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1fa00953e0 0x7f1fa0123570 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800e390 tx=0x7f1f9800e850 comp rx=0 tx=0).stop 2024-09-13T15:44:07.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.138+0000 7f1fa76b7640 1 -- 172.21.15.70:0/47394722 shutdown_connections 2024-09-13T15:44:07.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.138+0000 7f1fa76b7640 1 --2- 172.21.15.70:0/47394722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1fa00953e0 0x7f1fa0123570 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.138+0000 7f1fa76b7640 1 --2- 172.21.15.70:0/47394722 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1fa006ddf0 0x7f1fa0123030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.138+0000 7f1fa76b7640 1 --2- 172.21.15.70:0/47394722 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f1fa006cfc0 0x7f1fa0122af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.138+0000 7f1fa76b7640 1 -- 172.21.15.70:0/47394722 >> 172.21.15.70:0/47394722 conn(0x7f1fa0098b10 msgr2=0x7f1fa0126cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:07.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.139+0000 7f1fa76b7640 1 -- 172.21.15.70:0/47394722 shutdown_connections 2024-09-13T15:44:07.140 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.139+0000 7f1fa76b7640 1 -- 172.21.15.70:0/47394722 wait complete. 2024-09-13T15:44:07.150 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-13T15:44:05.540582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":128992,"kb_used_data":2208,"kb_used_omap":17,"kb_used_meta":126702,"kb_avail":377358368,"statfs":{"total":386547056640,"available":386414968832,"internally_reserved":0,"allocated":2260992,"data_stored":1554908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":129743093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001114"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599825+0000","last_change":"2024-09-13T15:43:56.791803+0000","last_active":"2024-09-13T15:44:01.599825+0000","last_peered":"2024-09-13T15:44:01.599825+0000","last_clean":"2024-09-13T15:44:01.599825+0000","last_became_active":"2024-09-13T15:43:56.791454+0000","last_became_peered":"2024-09-13T15:43:56.791454+0000","last_unstale":"2024-09-13T15:44:01.599825+0000","last_undegraded":"2024-09-13T15:44:01.599825+0000","last_fullsized":"2024-09-13T15:44:01.599825+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:38.078282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035985100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599965+0000","last_change":"2024-09-13T15:43:54.574295+0000","last_active":"2024-09-13T15:44:01.599965+0000","last_peered":"2024-09-13T15:44:01.599965+0000","last_clean":"2024-09-13T15:44:01.599965+0000","last_became_active":"2024-09-13T15:43:54.573978+0000","last_became_peered":"2024-09-13T15:43:54.573978+0000","last_unstale":"2024-09-13T15:44:01.599965+0000","last_undegraded":"2024-09-13T15:44:01.599965+0000","last_fullsized":"2024-09-13T15:44:01.599965+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:09.974326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054553999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599736+0000","last_change":"2024-09-13T15:43:56.792140+0000","last_active":"2024-09-13T15:44:01.599736+0000","last_peered":"2024-09-13T15:44:01.599736+0000","last_clean":"2024-09-13T15:44:01.599736+0000","last_became_active":"2024-09-13T15:43:56.791789+0000","last_became_peered":"2024-09-13T15:43:56.791789+0000","last_unstale":"2024-09-13T15:44:01.599736+0000","last_undegraded":"2024-09-13T15:44:01.599736+0000","last_fullsized":"2024-09-13T15:44:01.599736+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:51.347592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011766770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599970+0000","last_change":"2024-09-13T15:43:54.573858+0000","last_active":"2024-09-13T15:44:01.599970+0000","last_peered":"2024-09-13T15:44:01.599970+0000","last_clean":"2024-09-13T15:44:01.599970+0000","last_became_active":"2024-09-13T15:43:54.573540+0000","last_became_peered":"2024-09-13T15:43:54.573540+0000","last_unstale":"2024-09-13T15:44:01.599970+0000","last_undegraded":"2024-09-13T15:44:01.599970+0000","last_fullsized":"2024-09-13T15:44:01.599970+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:50.925568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024452499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599730+0000","last_change":"2024-09-13T15:43:56.584865+0000","last_active":"2024-09-13T15:44:01.599730+0000","last_peered":"2024-09-13T15:44:01.599730+0000","last_clean":"2024-09-13T15:44:01.599730+0000","last_became_active":"2024-09-13T15:43:56.584517+0000","last_became_peered":"2024-09-13T15:43:56.584517+0000","last_unstale":"2024-09-13T15:44:01.599730+0000","last_undegraded":"2024-09-13T15:44:01.599730+0000","last_fullsized":"2024-09-13T15:44:01.599730+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:29:26.245037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094542300000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T15:43:56.583705+0000","last_change":"2024-09-13T15:43:56.582866+0000","last_active":"2024-09-13T15:43:56.583705+0000","last_peered":"2024-09-13T15:43:56.583705+0000","last_clean":"2024-09-13T15:43:56.583705+0000","last_became_active":"2024-09-13T15:43:56.582550+0000","last_became_peered":"2024-09-13T15:43:56.582550+0000","last_unstale":"2024-09-13T15:43:56.583705+0000","last_undegraded":"2024-09-13T15:43:56.583705+0000","last_fullsized":"2024-09-13T15:43:56.583705+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:26.074766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089033199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599643+0000","last_change":"2024-09-13T15:43:56.585264+0000","last_active":"2024-09-13T15:44:01.599643+0000","last_peered":"2024-09-13T15:44:01.599643+0000","last_clean":"2024-09-13T15:44:01.599643+0000","last_became_active":"2024-09-13T15:43:56.584633+0000","last_became_peered":"2024-09-13T15:43:56.584633+0000","last_unstale":"2024-09-13T15:44:01.599643+0000","last_undegraded":"2024-09-13T15:44:01.599643+0000","last_fullsized":"2024-09-13T15:44:01.599643+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:29.252314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034895899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599737+0000","last_change":"2024-09-13T15:43:56.583583+0000","last_active":"2024-09-13T15:44:01.599737+0000","last_peered":"2024-09-13T15:44:01.599737+0000","last_clean":"2024-09-13T15:44:01.599737+0000","last_became_active":"2024-09-13T15:43:56.582549+0000","last_became_peered":"2024-09-13T15:43:56.582549+0000","last_unstale":"2024-09-13T15:44:01.599737+0000","last_undegraded":"2024-09-13T15:44:01.599737+0000","last_fullsized":"2024-09-13T15:44:01.599737+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:58:03.411133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276560000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'63","reported_seq":140,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599581+0000","last_change":"2024-09-13T15:43:54.574397+0000","last_active":"2024-09-13T15:44:01.599581+0000","last_peered":"2024-09-13T15:44:01.599581+0000","last_clean":"2024-09-13T15:44:01.599581+0000","last_became_active":"2024-09-13T15:43:54.574044+0000","last_became_peered":"2024-09-13T15:43:54.574044+0000","last_unstale":"2024-09-13T15:44:01.599581+0000","last_undegraded":"2024-09-13T15:44:01.599581+0000","last_fullsized":"2024-09-13T15:44:01.599581+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_clean_scrub_stamp":"2024-09-13T15:42:21.689345+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:48:10.619565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":28,"seq":120259084292,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":268,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":274432,"data_stored":95335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.215}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]}]},{"osd":2,"up_from":27,"seq":115964116996,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":256,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":262144,"data_stored":91751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]}]},{"osd":1,"up_from":26,"seq":111669149700,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":868352,"data_stored":685703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":32435774},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]}]},{"osd":0,"up_from":25,"seq":107374182404,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":836,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":856064,"data_stored":682119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T15:44:07.151 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T15:44:07.151 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T15:44:07.151 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T15:44:07.151 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T15:44:07.246 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.244+0000 7f799913e640 1 Processor -- start 2024-09-13T15:44:07.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.245+0000 7f799913e640 1 -- start start 2024-09-13T15:44:07.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.245+0000 7f799913e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f799406ddf0 0x7f799412f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.245+0000 7f799913e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f799412f960 0x7f7994131d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.245+0000 7f799913e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7994095ed0 con 0x7f79940953e0 2024-09-13T15:44:07.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.245+0000 7f799913e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7994097d40 con 0x7f799412f960 2024-09-13T15:44:07.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.245+0000 7f799913e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799406f110 con 0x7f799406ddf0 2024-09-13T15:44:07.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.245+0000 7f799259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f799406ddf0 0x7f799412f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.245+0000 7f7992d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f79940953e0 0x7f79940957c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55452/0 (socket says 172.21.15.70:55452) 2024-09-13T15:44:07.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.245+0000 7f7992d9d640 1 -- 172.21.15.70:0/664821659 learned_addr learned my addr 172.21.15.70:0/664821659 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:07.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.245+0000 7f799359e640 1 --2- 172.21.15.70:0/664821659 >> v2:172.21.15.70:3300/0 conn(0x7f799412f960 0x7f7994131d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.247 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.245+0000 7f799259c640 1 -- 172.21.15.70:0/664821659 >> v2:172.21.15.70:3300/0 conn(0x7f799412f960 msgr2=0x7f7994131d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:07.248 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.245+0000 7f799259c640 1 --2- 172.21.15.70:0/664821659 >> v2:172.21.15.70:3300/0 conn(0x7f799412f960 0x7f7994131d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.248 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.245+0000 7f7991d9b640 1 -- 172.21.15.70:0/664821659 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459026487 0 0) 0x7f7994095ed0 con 0x7f79940953e0 2024-09-13T15:44:07.248 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.245+0000 7f799259c640 1 -- 172.21.15.70:0/664821659 >> v1:172.21.15.70:6789/0 conn(0x7f79940953e0 legacy=0x7f79940957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.248 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.245+0000 7f799259c640 1 -- 172.21.15.70:0/664821659 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799406eb70 con 0x7f799406ddf0 2024-09-13T15:44:07.250 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.246+0000 7f799259c640 1 --2- 172.21.15.70:0/664821659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f799406ddf0 0x7f799412f420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f798800bfa0 tx=0x7f798802c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e356c09b1bac9f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:07.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.246+0000 7f7991d9b640 1 -- 172.21.15.70:0/664821659 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988009540 con 0x7f799406ddf0 2024-09-13T15:44:07.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.246+0000 7f7991d9b640 1 -- 172.21.15.70:0/664821659 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798802f050 con 0x7f799406ddf0 2024-09-13T15:44:07.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.246+0000 7f7991d9b640 1 -- 172.21.15.70:0/664821659 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79880098c0 con 0x7f799406ddf0 2024-09-13T15:44:07.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.247+0000 7f799913e640 1 -- 172.21.15.70:0/664821659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f799406ddf0 msgr2=0x7f799412f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.247+0000 7f799913e640 1 --2- 172.21.15.70:0/664821659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f799406ddf0 0x7f799412f420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f798800bfa0 tx=0x7f798802c750 comp rx=0 tx=0).stop 2024-09-13T15:44:07.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.247+0000 7f799913e640 1 -- 172.21.15.70:0/664821659 shutdown_connections 2024-09-13T15:44:07.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.247+0000 7f799913e640 1 --2- 172.21.15.70:0/664821659 >> v2:172.21.15.70:3300/0 conn(0x7f799412f960 0x7f7994131d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.247+0000 7f799913e640 1 --2- 172.21.15.70:0/664821659 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f799406ddf0 0x7f799412f420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.247+0000 7f799913e640 1 -- 172.21.15.70:0/664821659 >> 172.21.15.70:0/664821659 conn(0x7f7994098b10 msgr2=0x7f79940963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:07.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.247+0000 7f799913e640 1 -- 172.21.15.70:0/664821659 shutdown_connections 2024-09-13T15:44:07.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.247+0000 7f799913e640 1 -- 172.21.15.70:0/664821659 wait complete. 2024-09-13T15:44:07.251 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.248+0000 7f799913e640 1 Processor -- start 2024-09-13T15:44:07.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.248+0000 7f799913e640 1 -- start start 2024-09-13T15:44:07.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.248+0000 7f799913e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f799406ddf0 0x7f79941bdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.248+0000 7f799913e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79940953e0 0x7f79941be2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.248+0000 7f799913e640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f799412f960 0x7f79941be7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.248+0000 7f799913e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7994121da0 con 0x7f79940953e0 2024-09-13T15:44:07.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.248+0000 7f799913e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799412bb10 con 0x7f799406ddf0 2024-09-13T15:44:07.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.248+0000 7f799913e640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79941249f0 con 0x7f799412f960 2024-09-13T15:44:07.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.248+0000 7f7992d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f799406ddf0 0x7f79941bdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.248+0000 7f799259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79940953e0 0x7f79941be2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.248+0000 7f799259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79940953e0 0x7f79941be2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54136/0 (socket says 172.21.15.70:54136) 2024-09-13T15:44:07.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.248+0000 7f799259c640 1 -- 172.21.15.70:0/585096967 learned_addr learned my addr 172.21.15.70:0/585096967 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:07.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.248+0000 7f7992d9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f799406ddf0 0x7f79941bdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33236/0 (socket says 172.21.15.70:33236) 2024-09-13T15:44:07.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.248+0000 7f799359e640 1 --2- 172.21.15.70:0/585096967 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f799412f960 0x7f79941be7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.252 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.249+0000 7f7992d9d640 1 -- 172.21.15.70:0/585096967 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f799412f960 msgr2=0x7f79941be7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.249+0000 7f7992d9d640 1 --2- 172.21.15.70:0/585096967 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f799412f960 0x7f79941be7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.249+0000 7f7992d9d640 1 -- 172.21.15.70:0/585096967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79940953e0 msgr2=0x7f79941be2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.249+0000 7f7992d9d640 1 --2- 172.21.15.70:0/585096967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79940953e0 0x7f79941be2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.249+0000 7f7992d9d640 1 -- 172.21.15.70:0/585096967 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798800bc80 con 0x7f799406ddf0 2024-09-13T15:44:07.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.249+0000 7f799259c640 1 --2- 172.21.15.70:0/585096967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79940953e0 0x7f79941be2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:07.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.249+0000 7f7992d9d640 1 --2- 172.21.15.70:0/585096967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f799406ddf0 0x7f79941bdd60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f798c00c3e0 tx=0x7f798c00c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:07.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.249+0000 7f797b7fe640 1 -- 172.21.15.70:0/585096967 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c0074e0 con 0x7f799406ddf0 2024-09-13T15:44:07.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.249+0000 7f797b7fe640 1 -- 172.21.15.70:0/585096967 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798c007640 con 0x7f799406ddf0 2024-09-13T15:44:07.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.249+0000 7f797b7fe640 1 -- 172.21.15.70:0/585096967 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c004a20 con 0x7f799406ddf0 2024-09-13T15:44:07.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.249+0000 7f799913e640 1 -- 172.21.15.70:0/585096967 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799412b1a0 con 0x7f799406ddf0 2024-09-13T15:44:07.253 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.249+0000 7f799913e640 1 -- 172.21.15.70:0/585096967 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7994094080 con 0x7f799406ddf0 2024-09-13T15:44:07.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.250+0000 7f797b7fe640 1 -- 172.21.15.70:0/585096967 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f798c004b80 con 0x7f799406ddf0 2024-09-13T15:44:07.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.250+0000 7f799913e640 1 -- 172.21.15.70:0/585096967 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79940957c0 con 0x7f799406ddf0 2024-09-13T15:44:07.254 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.250+0000 7f797b7fe640 1 -- 172.21.15.70:0/585096967 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f798c05c620 con 0x7f799406ddf0 2024-09-13T15:44:07.255 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.254+0000 7f797b7fe640 1 -- 172.21.15.70:0/585096967 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f798c03a290 con 0x7f799406ddf0 2024-09-13T15:44:07.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.409+0000 7f799913e640 1 -- 172.21.15.70:0/585096967 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7994066fa0 con 0x7f799406ddf0 2024-09-13T15:44:07.410 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.409+0000 7f797b7fe640 1 -- 172.21.15.70:0/585096967 <== mon.1 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f798c03d8b0 con 0x7f799406ddf0 2024-09-13T15:44:07.411 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:44:07.411 INFO:teuthology.orchestra.run.smithi070.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T15:44:07.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.411+0000 7f799913e640 1 -- 172.21.15.70:0/585096967 >> v1:172.21.15.70:6816/1229510400 conn(0x7f7980047560 legacy=0x7f7980049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.411+0000 7f799913e640 1 -- 172.21.15.70:0/585096967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f799406ddf0 msgr2=0x7f79941bdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.411+0000 7f799913e640 1 --2- 172.21.15.70:0/585096967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f799406ddf0 0x7f79941bdd60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f798c00c3e0 tx=0x7f798c00c8a0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.411+0000 7f799913e640 1 -- 172.21.15.70:0/585096967 shutdown_connections 2024-09-13T15:44:07.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.411+0000 7f799913e640 1 --2- 172.21.15.70:0/585096967 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f799412f960 0x7f79941be7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.411+0000 7f799913e640 1 --2- 172.21.15.70:0/585096967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79940953e0 0x7f79941be2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.411+0000 7f799913e640 1 --2- 172.21.15.70:0/585096967 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f799406ddf0 0x7f79941bdd60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.411+0000 7f799913e640 1 -- 172.21.15.70:0/585096967 >> 172.21.15.70:0/585096967 conn(0x7f7994098b10 msgr2=0x7f7994126cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:07.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.411+0000 7f799913e640 1 -- 172.21.15.70:0/585096967 shutdown_connections 2024-09-13T15:44:07.413 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.411+0000 7f799913e640 1 -- 172.21.15.70:0/585096967 wait complete. 2024-09-13T15:44:07.423 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T15:44:07.423 INFO:teuthology.run_tasks:Running task print... 2024-09-13T15:44:07.433 INFO:teuthology.task.print:**** done ceph.restart 2024-09-13T15:44:07.433 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2024-09-13T15:44:07.449 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2024-09-13T15:44:07.458 DEBUG:teuthology.run_tasks:Unwinding manager install.upgrade 2024-09-13T15:44:07.467 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T15:44:07.476 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T15:44:07.476 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T15:44:07.581 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.573+0000 7fb05d7ff640 1 Processor -- start 2024-09-13T15:44:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.573+0000 7fb05d7ff640 1 -- start start 2024-09-13T15:44:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.574+0000 7fb05d7ff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb05812a2a0 0x7fb05812a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.574+0000 7fb05d7ff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb05806be70 0x7fb05806c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.574+0000 7fb05d7ff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb058129e50 con 0x7fb05806d000 2024-09-13T15:44:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.574+0000 7fb05d7ff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0580a04c0 con 0x7fb05806be70 2024-09-13T15:44:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.574+0000 7fb05d7ff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05812a080 con 0x7fb05812a2a0 2024-09-13T15:44:07.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.574+0000 7fb056ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb05812a2a0 0x7fb05812a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.574+0000 7fb0567fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb05806d000 0x7fb05806b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55456/0 (socket says 172.21.15.70:55456) 2024-09-13T15:44:07.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.574+0000 7fb0567fc640 1 -- 172.21.15.70:0/2501215131 learned_addr learned my addr 172.21.15.70:0/2501215131 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:07.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.574+0000 7fb0577fe640 1 --2- 172.21.15.70:0/2501215131 >> v2:172.21.15.70:3300/0 conn(0x7fb05806be70 0x7fb05806c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.574+0000 7fb056ffd640 1 -- 172.21.15.70:0/2501215131 >> v2:172.21.15.70:3300/0 conn(0x7fb05806be70 msgr2=0x7fb05806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:07.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.574+0000 7fb056ffd640 1 --2- 172.21.15.70:0/2501215131 >> v2:172.21.15.70:3300/0 conn(0x7fb05806be70 0x7fb05806c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.583 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.574+0000 7fb056ffd640 1 -- 172.21.15.70:0/2501215131 >> v1:172.21.15.70:6789/0 conn(0x7fb05806d000 legacy=0x7fb05806b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.574+0000 7fb056ffd640 1 -- 172.21.15.70:0/2501215131 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05806ca00 con 0x7fb05812a2a0 2024-09-13T15:44:07.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.574+0000 7fb055ffb640 1 -- 172.21.15.70:0/2501215131 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502570104 0 0) 0x7fb058129e50 con 0x7fb05806d000 2024-09-13T15:44:07.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.574+0000 7fb0577fe640 1 --2- 172.21.15.70:0/2501215131 >> v2:172.21.15.70:3300/0 conn(0x7fb05806be70 0x7fb05806c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:07.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.575+0000 7fb056ffd640 1 --2- 172.21.15.70:0/2501215131 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb05812a2a0 0x7fb05812a680 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb050009d90 tx=0x7fb05002a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46ce69eb97df6fcd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:07.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.575+0000 7fb055ffb640 1 -- 172.21.15.70:0/2501215131 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050007740 con 0x7fb05812a2a0 2024-09-13T15:44:07.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.575+0000 7fb055ffb640 1 -- 172.21.15.70:0/2501215131 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05002d050 con 0x7fb05812a2a0 2024-09-13T15:44:07.584 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.575+0000 7fb055ffb640 1 -- 172.21.15.70:0/2501215131 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050007ac0 con 0x7fb05812a2a0 2024-09-13T15:44:07.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.575+0000 7fb05d7ff640 1 -- 172.21.15.70:0/2501215131 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb05812a2a0 msgr2=0x7fb05812a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.575+0000 7fb05d7ff640 1 --2- 172.21.15.70:0/2501215131 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb05812a2a0 0x7fb05812a680 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb050009d90 tx=0x7fb05002a530 comp rx=0 tx=0).stop 2024-09-13T15:44:07.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.576+0000 7fb05d7ff640 1 -- 172.21.15.70:0/2501215131 shutdown_connections 2024-09-13T15:44:07.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.576+0000 7fb05d7ff640 1 --2- 172.21.15.70:0/2501215131 >> v2:172.21.15.70:3300/0 conn(0x7fb05806be70 0x7fb05806c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.576+0000 7fb05d7ff640 1 --2- 172.21.15.70:0/2501215131 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb05812a2a0 0x7fb05812a680 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.576+0000 7fb05d7ff640 1 -- 172.21.15.70:0/2501215131 >> 172.21.15.70:0/2501215131 conn(0x7fb05809e040 msgr2=0x7fb05809fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:07.585 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.576+0000 7fb05d7ff640 1 -- 172.21.15.70:0/2501215131 shutdown_connections 2024-09-13T15:44:07.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.576+0000 7fb05d7ff640 1 -- 172.21.15.70:0/2501215131 wait complete. 2024-09-13T15:44:07.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb05d7ff640 1 Processor -- start 2024-09-13T15:44:07.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb05d7ff640 1 -- start start 2024-09-13T15:44:07.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb05d7ff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb05806be70 0x7fb0581309a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb05d7ff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb05806d000 0x7fb058130ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb05d7ff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb05812a2a0 0x7fb058131420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.586 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb05d7ff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb058133d60 con 0x7fb05806be70 2024-09-13T15:44:07.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb05d7ff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0581341e0 con 0x7fb05806d000 2024-09-13T15:44:07.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb05d7ff640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb058132d40 con 0x7fb05812a2a0 2024-09-13T15:44:07.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb056ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb05806be70 0x7fb0581309a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb0567fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb05806d000 0x7fb058130ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb056ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb05806be70 0x7fb0581309a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54150/0 (socket says 172.21.15.70:54150) 2024-09-13T15:44:07.587 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb056ffd640 1 -- 172.21.15.70:0/3016603160 learned_addr learned my addr 172.21.15.70:0/3016603160 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:07.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb0567fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb05806d000 0x7fb058130ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33264/0 (socket says 172.21.15.70:33264) 2024-09-13T15:44:07.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb0577fe640 1 --2- 172.21.15.70:0/3016603160 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb05812a2a0 0x7fb058131420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb0567fc640 1 -- 172.21.15.70:0/3016603160 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb05812a2a0 msgr2=0x7fb058131420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb0567fc640 1 --2- 172.21.15.70:0/3016603160 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb05812a2a0 0x7fb058131420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb0567fc640 1 -- 172.21.15.70:0/3016603160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb05806be70 msgr2=0x7fb0581309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb0567fc640 1 --2- 172.21.15.70:0/3016603160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb05806be70 0x7fb0581309a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb0567fc640 1 -- 172.21.15.70:0/3016603160 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb048009400 con 0x7fb05806d000 2024-09-13T15:44:07.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.577+0000 7fb056ffd640 1 --2- 172.21.15.70:0/3016603160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb05806be70 0x7fb0581309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:07.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.578+0000 7fb0567fc640 1 --2- 172.21.15.70:0/3016603160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb05806d000 0x7fb058130ee0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb04000c3e0 tx=0x7fb04000c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:07.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.578+0000 7fb04f7fe640 1 -- 172.21.15.70:0/3016603160 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0400074e0 con 0x7fb05806d000 2024-09-13T15:44:07.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.578+0000 7fb04f7fe640 1 -- 172.21.15.70:0/3016603160 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb040007640 con 0x7fb05806d000 2024-09-13T15:44:07.588 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.578+0000 7fb05d7ff640 1 -- 172.21.15.70:0/3016603160 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb050009a70 con 0x7fb05806d000 2024-09-13T15:44:07.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.578+0000 7fb04f7fe640 1 -- 172.21.15.70:0/3016603160 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040004a20 con 0x7fb05806d000 2024-09-13T15:44:07.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.578+0000 7fb05d7ff640 1 -- 172.21.15.70:0/3016603160 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb058133760 con 0x7fb05806d000 2024-09-13T15:44:07.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.579+0000 7fb05d7ff640 1 -- 172.21.15.70:0/3016603160 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb058067400 con 0x7fb05806d000 2024-09-13T15:44:07.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.579+0000 7fb04f7fe640 1 -- 172.21.15.70:0/3016603160 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fb040004b80 con 0x7fb05806d000 2024-09-13T15:44:07.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.579+0000 7fb04f7fe640 1 -- 172.21.15.70:0/3016603160 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fb04005c540 con 0x7fb05806d000 2024-09-13T15:44:07.589 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.583+0000 7fb04f7fe640 1 -- 172.21.15.70:0/3016603160 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fb04003a1b0 con 0x7fb05806d000 2024-09-13T15:44:07.716 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.715+0000 7fb05d7ff640 1 -- 172.21.15.70:0/3016603160 --> v1:172.21.15.70:6816/1229510400 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0580008d0 con 0x7fb038047590 2024-09-13T15:44:07.717 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.716+0000 7fb04f7fe640 1 -- 172.21.15.70:0/3016603160 <== mgr.24114 v1:172.21.15.70:6816/1229510400 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34750 (unknown 2965378022 0 4254729499) 0x7fb0580008d0 con 0x7fb038047590 2024-09-13T15:44:07.718 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:44:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-13T15:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.718+0000 7fb05d7ff640 1 -- 172.21.15.70:0/3016603160 >> v1:172.21.15.70:6816/1229510400 conn(0x7fb038047590 legacy=0x7fb038049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.718+0000 7fb05d7ff640 1 -- 172.21.15.70:0/3016603160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb05806d000 msgr2=0x7fb058130ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.718+0000 7fb05d7ff640 1 --2- 172.21.15.70:0/3016603160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb05806d000 0x7fb058130ee0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb04000c3e0 tx=0x7fb04000c7e0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.718+0000 7fb05d7ff640 1 -- 172.21.15.70:0/3016603160 shutdown_connections 2024-09-13T15:44:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.718+0000 7fb05d7ff640 1 --2- 172.21.15.70:0/3016603160 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fb05812a2a0 0x7fb058131420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.718+0000 7fb05d7ff640 1 --2- 172.21.15.70:0/3016603160 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb05806d000 0x7fb058130ee0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.718+0000 7fb05d7ff640 1 --2- 172.21.15.70:0/3016603160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb05806be70 0x7fb0581309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.718+0000 7fb05d7ff640 1 -- 172.21.15.70:0/3016603160 >> 172.21.15.70:0/3016603160 conn(0x7fb05809e040 msgr2=0x7fb058126f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.718+0000 7fb05d7ff640 1 -- 172.21.15.70:0/3016603160 shutdown_connections 2024-09-13T15:44:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.718+0000 7fb05d7ff640 1 -- 172.21.15.70:0/3016603160 wait complete. 2024-09-13T15:44:07.729 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-13T15:44:07.541008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":128996,"kb_used_data":2216,"kb_used_omap":17,"kb_used_meta":126702,"kb_avail":377358364,"statfs":{"total":386547056640,"available":386414964736,"internally_reserved":0,"allocated":2269184,"data_stored":1558473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":129743093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.945829"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599825+0000","last_change":"2024-09-13T15:43:56.791803+0000","last_active":"2024-09-13T15:44:01.599825+0000","last_peered":"2024-09-13T15:44:01.599825+0000","last_clean":"2024-09-13T15:44:01.599825+0000","last_became_active":"2024-09-13T15:43:56.791454+0000","last_became_peered":"2024-09-13T15:43:56.791454+0000","last_unstale":"2024-09-13T15:44:01.599825+0000","last_undegraded":"2024-09-13T15:44:01.599825+0000","last_fullsized":"2024-09-13T15:44:01.599825+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:38.078282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035985100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599965+0000","last_change":"2024-09-13T15:43:54.574295+0000","last_active":"2024-09-13T15:44:01.599965+0000","last_peered":"2024-09-13T15:44:01.599965+0000","last_clean":"2024-09-13T15:44:01.599965+0000","last_became_active":"2024-09-13T15:43:54.573978+0000","last_became_peered":"2024-09-13T15:43:54.573978+0000","last_unstale":"2024-09-13T15:44:01.599965+0000","last_undegraded":"2024-09-13T15:44:01.599965+0000","last_fullsized":"2024-09-13T15:44:01.599965+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:09.974326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054553999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599736+0000","last_change":"2024-09-13T15:43:56.792140+0000","last_active":"2024-09-13T15:44:01.599736+0000","last_peered":"2024-09-13T15:44:01.599736+0000","last_clean":"2024-09-13T15:44:01.599736+0000","last_became_active":"2024-09-13T15:43:56.791789+0000","last_became_peered":"2024-09-13T15:43:56.791789+0000","last_unstale":"2024-09-13T15:44:01.599736+0000","last_undegraded":"2024-09-13T15:44:01.599736+0000","last_fullsized":"2024-09-13T15:44:01.599736+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:51.347592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011766770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599970+0000","last_change":"2024-09-13T15:43:54.573858+0000","last_active":"2024-09-13T15:44:01.599970+0000","last_peered":"2024-09-13T15:44:01.599970+0000","last_clean":"2024-09-13T15:44:01.599970+0000","last_became_active":"2024-09-13T15:43:54.573540+0000","last_became_peered":"2024-09-13T15:43:54.573540+0000","last_unstale":"2024-09-13T15:44:01.599970+0000","last_undegraded":"2024-09-13T15:44:01.599970+0000","last_fullsized":"2024-09-13T15:44:01.599970+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:50.925568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024452499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599730+0000","last_change":"2024-09-13T15:43:56.584865+0000","last_active":"2024-09-13T15:44:01.599730+0000","last_peered":"2024-09-13T15:44:01.599730+0000","last_clean":"2024-09-13T15:44:01.599730+0000","last_became_active":"2024-09-13T15:43:56.584517+0000","last_became_peered":"2024-09-13T15:43:56.584517+0000","last_unstale":"2024-09-13T15:44:01.599730+0000","last_undegraded":"2024-09-13T15:44:01.599730+0000","last_fullsized":"2024-09-13T15:44:01.599730+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:29:26.245037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094542300000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T15:43:56.583705+0000","last_change":"2024-09-13T15:43:56.582866+0000","last_active":"2024-09-13T15:43:56.583705+0000","last_peered":"2024-09-13T15:43:56.583705+0000","last_clean":"2024-09-13T15:43:56.583705+0000","last_became_active":"2024-09-13T15:43:56.582550+0000","last_became_peered":"2024-09-13T15:43:56.582550+0000","last_unstale":"2024-09-13T15:43:56.583705+0000","last_undegraded":"2024-09-13T15:43:56.583705+0000","last_fullsized":"2024-09-13T15:43:56.583705+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:26.074766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089033199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599643+0000","last_change":"2024-09-13T15:43:56.585264+0000","last_active":"2024-09-13T15:44:01.599643+0000","last_peered":"2024-09-13T15:44:01.599643+0000","last_clean":"2024-09-13T15:44:01.599643+0000","last_became_active":"2024-09-13T15:43:56.584633+0000","last_became_peered":"2024-09-13T15:43:56.584633+0000","last_unstale":"2024-09-13T15:44:01.599643+0000","last_undegraded":"2024-09-13T15:44:01.599643+0000","last_fullsized":"2024-09-13T15:44:01.599643+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:29.252314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034895899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599737+0000","last_change":"2024-09-13T15:43:56.583583+0000","last_active":"2024-09-13T15:44:01.599737+0000","last_peered":"2024-09-13T15:44:01.599737+0000","last_clean":"2024-09-13T15:44:01.599737+0000","last_became_active":"2024-09-13T15:43:56.582549+0000","last_became_peered":"2024-09-13T15:43:56.582549+0000","last_unstale":"2024-09-13T15:44:01.599737+0000","last_undegraded":"2024-09-13T15:44:01.599737+0000","last_fullsized":"2024-09-13T15:44:01.599737+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:58:03.411133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276560000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'63","reported_seq":140,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599581+0000","last_change":"2024-09-13T15:43:54.574397+0000","last_active":"2024-09-13T15:44:01.599581+0000","last_peered":"2024-09-13T15:44:01.599581+0000","last_clean":"2024-09-13T15:44:01.599581+0000","last_became_active":"2024-09-13T15:43:54.574044+0000","last_became_peered":"2024-09-13T15:43:54.574044+0000","last_unstale":"2024-09-13T15:44:01.599581+0000","last_undegraded":"2024-09-13T15:44:01.599581+0000","last_fullsized":"2024-09-13T15:44:01.599581+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_clean_scrub_stamp":"2024-09-13T15:42:21.689345+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:48:10.619565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":28,"seq":120259084292,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":268,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":274432,"data_stored":95335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.215}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]}]},{"osd":2,"up_from":27,"seq":115964116996,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":256,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":262144,"data_stored":91751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]}]},{"osd":1,"up_from":26,"seq":111669149700,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":868352,"data_stored":685703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":32435774},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]}]},{"osd":0,"up_from":25,"seq":107374182405,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":844,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":864256,"data_stored":685684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.245}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T15:44:07.730 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T15:44:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.826+0000 7fdea6de9640 1 Processor -- start 2024-09-13T15:44:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.826+0000 7fdea6de9640 1 -- start start 2024-09-13T15:44:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea6de9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdea0129de0 0x7fdea012a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea6de9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdea012a700 0x7fdea006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea6de9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdea006d520 0x7fdea006beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.829 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea6de9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea006e3f0 con 0x7fdea012a700 2024-09-13T15:44:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea6de9640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea006e920 con 0x7fdea0129de0 2024-09-13T15:44:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea6de9640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdea006ec50 con 0x7fdea006d520 2024-09-13T15:44:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea4b85640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdea0129de0 0x7fdea012a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea5386640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdea006d520 0x7fdea006beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fde97fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdea012a700 0x7fdea006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea4b85640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdea0129de0 0x7fdea012a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33266/0 (socket says 172.21.15.70:33266) 2024-09-13T15:44:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea4b85640 1 -- 172.21.15.70:0/2257219270 learned_addr learned my addr 172.21.15.70:0/2257219270 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea5386640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdea006d520 0x7fdea006beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:39716/0 (socket says 172.21.15.70:39716) 2024-09-13T15:44:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea5386640 1 -- 172.21.15.70:0/2257219270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdea0129de0 msgr2=0x7fdea012a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea5386640 1 --2- 172.21.15.70:0/2257219270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdea0129de0 0x7fdea012a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea5386640 1 -- 172.21.15.70:0/2257219270 >> v2:172.21.15.70:3300/0 conn(0x7fdea012a700 msgr2=0x7fdea006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea5386640 1 --2- 172.21.15.70:0/2257219270 >> v2:172.21.15.70:3300/0 conn(0x7fdea012a700 0x7fdea006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.830 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea5386640 1 -- 172.21.15.70:0/2257219270 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea006c3f0 con 0x7fdea006d520 2024-09-13T15:44:07.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fdea4b85640 1 --2- 172.21.15.70:0/2257219270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdea0129de0 0x7fdea012a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:07.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.827+0000 7fde97fff640 1 --2- 172.21.15.70:0/2257219270 >> v2:172.21.15.70:3300/0 conn(0x7fdea012a700 0x7fdea006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:07.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.828+0000 7fdea5386640 1 --2- 172.21.15.70:0/2257219270 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdea006d520 0x7fdea006beb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fde9800aea0 tx=0x7fde9802b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=448d52e9a248c0e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:07.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.828+0000 7fde977fe640 1 -- 172.21.15.70:0/2257219270 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98008640 con 0x7fdea006d520 2024-09-13T15:44:07.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.828+0000 7fde977fe640 1 -- 172.21.15.70:0/2257219270 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9802e050 con 0x7fdea006d520 2024-09-13T15:44:07.831 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.828+0000 7fde977fe640 1 -- 172.21.15.70:0/2257219270 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde980089c0 con 0x7fdea006d520 2024-09-13T15:44:07.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.828+0000 7fdea6de9640 1 -- 172.21.15.70:0/2257219270 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdea006d520 msgr2=0x7fdea006beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.828+0000 7fdea6de9640 1 --2- 172.21.15.70:0/2257219270 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdea006d520 0x7fdea006beb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fde9800aea0 tx=0x7fde9802b670 comp rx=0 tx=0).stop 2024-09-13T15:44:07.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.828+0000 7fdea6de9640 1 -- 172.21.15.70:0/2257219270 shutdown_connections 2024-09-13T15:44:07.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.828+0000 7fdea6de9640 1 --2- 172.21.15.70:0/2257219270 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdea006d520 0x7fdea006beb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.828+0000 7fdea6de9640 1 --2- 172.21.15.70:0/2257219270 >> v2:172.21.15.70:3300/0 conn(0x7fdea012a700 0x7fdea006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.828+0000 7fdea6de9640 1 --2- 172.21.15.70:0/2257219270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdea0129de0 0x7fdea012a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.828+0000 7fdea6de9640 1 -- 172.21.15.70:0/2257219270 >> 172.21.15.70:0/2257219270 conn(0x7fdea0098b10 msgr2=0x7fdea00963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:07.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.828+0000 7fdea6de9640 1 -- 172.21.15.70:0/2257219270 shutdown_connections 2024-09-13T15:44:07.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.829+0000 7fdea6de9640 1 -- 172.21.15.70:0/2257219270 wait complete. 2024-09-13T15:44:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.829+0000 7fdea6de9640 1 Processor -- start 2024-09-13T15:44:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.829+0000 7fdea6de9640 1 -- start start 2024-09-13T15:44:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea6de9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdea006d520 0x7fdea008aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea6de9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdea0129de0 0x7fdea008b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea6de9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdea012a700 0x7fdea008ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea6de9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea008cf00 con 0x7fdea012a700 2024-09-13T15:44:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea6de9640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea00712e0 con 0x7fdea0129de0 2024-09-13T15:44:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea6de9640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdea0129840 con 0x7fdea006d520 2024-09-13T15:44:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea4b85640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdea006d520 0x7fdea008aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea5386640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdea012a700 0x7fdea008ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea4b85640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdea006d520 0x7fdea008aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:39724/0 (socket says 172.21.15.70:39724) 2024-09-13T15:44:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fde97fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdea0129de0 0x7fdea008b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:07.835 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea4b85640 1 -- 172.21.15.70:0/418143389 learned_addr learned my addr 172.21.15.70:0/418143389 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea5386640 1 -- 172.21.15.70:0/418143389 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdea006d520 msgr2=0x7fdea008aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea5386640 1 --2- 172.21.15.70:0/418143389 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdea006d520 0x7fdea008aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea5386640 1 -- 172.21.15.70:0/418143389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdea0129de0 msgr2=0x7fdea008b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea5386640 1 --2- 172.21.15.70:0/418143389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdea0129de0 0x7fdea008b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea5386640 1 -- 172.21.15.70:0/418143389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c0094d0 con 0x7fdea012a700 2024-09-13T15:44:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fde97fff640 1 --2- 172.21.15.70:0/418143389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdea0129de0 0x7fdea008b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea4b85640 1 --2- 172.21.15.70:0/418143389 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdea006d520 0x7fdea008aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.830+0000 7fdea5386640 1 --2- 172.21.15.70:0/418143389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdea012a700 0x7fdea008ba70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fde98031000 tx=0x7fde98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.831+0000 7fde957fa640 1 -- 172.21.15.70:0/418143389 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9803d850 con 0x7fdea012a700 2024-09-13T15:44:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.831+0000 7fde957fa640 1 -- 172.21.15.70:0/418143389 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9803d9d0 con 0x7fdea012a700 2024-09-13T15:44:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.831+0000 7fdea6de9640 1 -- 172.21.15.70:0/418143389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde90009400 con 0x7fdea012a700 2024-09-13T15:44:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.831+0000 7fde957fa640 1 -- 172.21.15.70:0/418143389 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9803dd80 con 0x7fdea012a700 2024-09-13T15:44:07.836 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.831+0000 7fdea6de9640 1 -- 172.21.15.70:0/418143389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9800ab80 con 0x7fdea012a700 2024-09-13T15:44:07.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.832+0000 7fde957fa640 1 -- 172.21.15.70:0/418143389 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fde98039040 con 0x7fdea012a700 2024-09-13T15:44:07.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.832+0000 7fdea6de9640 1 -- 172.21.15.70:0/418143389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea012a1c0 con 0x7fdea012a700 2024-09-13T15:44:07.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.832+0000 7fde957fa640 1 -- 172.21.15.70:0/418143389 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fde9807d2a0 con 0x7fdea012a700 2024-09-13T15:44:07.837 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.835+0000 7fde957fa640 1 -- 172.21.15.70:0/418143389 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fde9805ea60 con 0x7fdea012a700 2024-09-13T15:44:07.969 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.968+0000 7fdea6de9640 1 -- 172.21.15.70:0/418143389 --> v1:172.21.15.70:6816/1229510400 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdea0097460 con 0x7fde8c047560 2024-09-13T15:44:07.970 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.969+0000 7fde957fa640 1 -- 172.21.15.70:0/418143389 <== mgr.24114 v1:172.21.15.70:6816/1229510400 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34750 (unknown 2965378022 0 4254729499) 0x7fdea0097460 con 0x7fde8c047560 2024-09-13T15:44:07.971 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:44:07.971 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-13T15:44:07.972 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.970+0000 7fdea6de9640 1 -- 172.21.15.70:0/418143389 >> v1:172.21.15.70:6816/1229510400 conn(0x7fde8c047560 legacy=0x7fde8c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.972 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.970+0000 7fdea6de9640 1 -- 172.21.15.70:0/418143389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdea012a700 msgr2=0x7fdea008ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:07.972 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.970+0000 7fdea6de9640 1 --2- 172.21.15.70:0/418143389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdea012a700 0x7fdea008ba70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fde98031000 tx=0x7fde98004a40 comp rx=0 tx=0).stop 2024-09-13T15:44:07.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.970+0000 7fdea6de9640 1 -- 172.21.15.70:0/418143389 shutdown_connections 2024-09-13T15:44:07.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.970+0000 7fdea6de9640 1 --2- 172.21.15.70:0/418143389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdea012a700 0x7fdea008ba70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.970+0000 7fdea6de9640 1 --2- 172.21.15.70:0/418143389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdea0129de0 0x7fdea008b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.970+0000 7fdea6de9640 1 --2- 172.21.15.70:0/418143389 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fdea006d520 0x7fdea008aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:07.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.970+0000 7fdea6de9640 1 -- 172.21.15.70:0/418143389 >> 172.21.15.70:0/418143389 conn(0x7fdea0098b10 msgr2=0x7fdea006ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:07.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.970+0000 7fdea6de9640 1 -- 172.21.15.70:0/418143389 shutdown_connections 2024-09-13T15:44:07.973 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:07.970+0000 7fdea6de9640 1 -- 172.21.15.70:0/418143389 wait complete. 2024-09-13T15:44:07.983 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-13T15:44:07.541008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":128996,"kb_used_data":2216,"kb_used_omap":17,"kb_used_meta":126702,"kb_avail":377358364,"statfs":{"total":386547056640,"available":386414964736,"internally_reserved":0,"allocated":2269184,"data_stored":1558473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":129743093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.945829"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599825+0000","last_change":"2024-09-13T15:43:56.791803+0000","last_active":"2024-09-13T15:44:01.599825+0000","last_peered":"2024-09-13T15:44:01.599825+0000","last_clean":"2024-09-13T15:44:01.599825+0000","last_became_active":"2024-09-13T15:43:56.791454+0000","last_became_peered":"2024-09-13T15:43:56.791454+0000","last_unstale":"2024-09-13T15:44:01.599825+0000","last_undegraded":"2024-09-13T15:44:01.599825+0000","last_fullsized":"2024-09-13T15:44:01.599825+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:38.078282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035985100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599965+0000","last_change":"2024-09-13T15:43:54.574295+0000","last_active":"2024-09-13T15:44:01.599965+0000","last_peered":"2024-09-13T15:44:01.599965+0000","last_clean":"2024-09-13T15:44:01.599965+0000","last_became_active":"2024-09-13T15:43:54.573978+0000","last_became_peered":"2024-09-13T15:43:54.573978+0000","last_unstale":"2024-09-13T15:44:01.599965+0000","last_undegraded":"2024-09-13T15:44:01.599965+0000","last_fullsized":"2024-09-13T15:44:01.599965+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:09.974326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054553999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599736+0000","last_change":"2024-09-13T15:43:56.792140+0000","last_active":"2024-09-13T15:44:01.599736+0000","last_peered":"2024-09-13T15:44:01.599736+0000","last_clean":"2024-09-13T15:44:01.599736+0000","last_became_active":"2024-09-13T15:43:56.791789+0000","last_became_peered":"2024-09-13T15:43:56.791789+0000","last_unstale":"2024-09-13T15:44:01.599736+0000","last_undegraded":"2024-09-13T15:44:01.599736+0000","last_fullsized":"2024-09-13T15:44:01.599736+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:51.347592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011766770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599970+0000","last_change":"2024-09-13T15:43:54.573858+0000","last_active":"2024-09-13T15:44:01.599970+0000","last_peered":"2024-09-13T15:44:01.599970+0000","last_clean":"2024-09-13T15:44:01.599970+0000","last_became_active":"2024-09-13T15:43:54.573540+0000","last_became_peered":"2024-09-13T15:43:54.573540+0000","last_unstale":"2024-09-13T15:44:01.599970+0000","last_undegraded":"2024-09-13T15:44:01.599970+0000","last_fullsized":"2024-09-13T15:44:01.599970+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:50.925568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024452499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599730+0000","last_change":"2024-09-13T15:43:56.584865+0000","last_active":"2024-09-13T15:44:01.599730+0000","last_peered":"2024-09-13T15:44:01.599730+0000","last_clean":"2024-09-13T15:44:01.599730+0000","last_became_active":"2024-09-13T15:43:56.584517+0000","last_became_peered":"2024-09-13T15:43:56.584517+0000","last_unstale":"2024-09-13T15:44:01.599730+0000","last_undegraded":"2024-09-13T15:44:01.599730+0000","last_fullsized":"2024-09-13T15:44:01.599730+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:29:26.245037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094542300000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T15:43:56.583705+0000","last_change":"2024-09-13T15:43:56.582866+0000","last_active":"2024-09-13T15:43:56.583705+0000","last_peered":"2024-09-13T15:43:56.583705+0000","last_clean":"2024-09-13T15:43:56.583705+0000","last_became_active":"2024-09-13T15:43:56.582550+0000","last_became_peered":"2024-09-13T15:43:56.582550+0000","last_unstale":"2024-09-13T15:43:56.583705+0000","last_undegraded":"2024-09-13T15:43:56.583705+0000","last_fullsized":"2024-09-13T15:43:56.583705+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:26.074766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089033199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599643+0000","last_change":"2024-09-13T15:43:56.585264+0000","last_active":"2024-09-13T15:44:01.599643+0000","last_peered":"2024-09-13T15:44:01.599643+0000","last_clean":"2024-09-13T15:44:01.599643+0000","last_became_active":"2024-09-13T15:43:56.584633+0000","last_became_peered":"2024-09-13T15:43:56.584633+0000","last_unstale":"2024-09-13T15:44:01.599643+0000","last_undegraded":"2024-09-13T15:44:01.599643+0000","last_fullsized":"2024-09-13T15:44:01.599643+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:29.252314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034895899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599737+0000","last_change":"2024-09-13T15:43:56.583583+0000","last_active":"2024-09-13T15:44:01.599737+0000","last_peered":"2024-09-13T15:44:01.599737+0000","last_clean":"2024-09-13T15:44:01.599737+0000","last_became_active":"2024-09-13T15:43:56.582549+0000","last_became_peered":"2024-09-13T15:43:56.582549+0000","last_unstale":"2024-09-13T15:44:01.599737+0000","last_undegraded":"2024-09-13T15:44:01.599737+0000","last_fullsized":"2024-09-13T15:44:01.599737+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:58:03.411133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276560000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'63","reported_seq":140,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599581+0000","last_change":"2024-09-13T15:43:54.574397+0000","last_active":"2024-09-13T15:44:01.599581+0000","last_peered":"2024-09-13T15:44:01.599581+0000","last_clean":"2024-09-13T15:44:01.599581+0000","last_became_active":"2024-09-13T15:43:54.574044+0000","last_became_peered":"2024-09-13T15:43:54.574044+0000","last_unstale":"2024-09-13T15:44:01.599581+0000","last_undegraded":"2024-09-13T15:44:01.599581+0000","last_fullsized":"2024-09-13T15:44:01.599581+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_clean_scrub_stamp":"2024-09-13T15:42:21.689345+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:48:10.619565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":28,"seq":120259084292,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":268,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":274432,"data_stored":95335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.215}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]}]},{"osd":2,"up_from":27,"seq":115964116996,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":256,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":262144,"data_stored":91751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]}]},{"osd":1,"up_from":26,"seq":111669149700,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":868352,"data_stored":685703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":32435774},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]}]},{"osd":0,"up_from":25,"seq":107374182405,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":844,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":864256,"data_stored":685684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.245}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T15:44:07.983 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T15:44:07.984 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T15:44:08.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.080+0000 7f0b81353640 1 Processor -- start 2024-09-13T15:44:08.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.080+0000 7f0b81353640 1 -- start start 2024-09-13T15:44:08.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.080+0000 7f0b81353640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0b7c06bc00 0x7f0b7c06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:08.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.080+0000 7f0b81353640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0b7c06c520 0x7f0b7c12d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:08.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.080+0000 7f0b81353640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b7c12d740 0x7f0b7c12dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:08.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.080+0000 7f0b81353640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b7c06c920 con 0x7f0b7c06bc00 2024-09-13T15:44:08.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.080+0000 7f0b81353640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c097940 con 0x7f0b7c12d740 2024-09-13T15:44:08.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.080+0000 7f0b81353640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c12e110 con 0x7f0b7c06c520 2024-09-13T15:44:08.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.080+0000 7f0b73fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0b7c06c520 0x7f0b7c12d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:08.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.080+0000 7f0b7affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0b7c06bc00 0x7f0b7c06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:08.083 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.080+0000 7f0b73fff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0b7c06c520 0x7f0b7c12d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:39738/0 (socket says 172.21.15.70:39738) 2024-09-13T15:44:08.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.080+0000 7f0b7affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0b7c06bc00 0x7f0b7c06bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54178/0 (socket says 172.21.15.70:54178) 2024-09-13T15:44:08.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.080+0000 7f0b73fff640 1 -- 172.21.15.70:0/1832668058 learned_addr learned my addr 172.21.15.70:0/1832668058 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:08.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.080+0000 7f0b7b7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b7c12d740 0x7f0b7c12dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:08.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.081+0000 7f0b73fff640 1 -- 172.21.15.70:0/1832668058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b7c12d740 msgr2=0x7f0b7c12dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:08.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.081+0000 7f0b73fff640 1 --2- 172.21.15.70:0/1832668058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b7c12d740 0x7f0b7c12dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.081+0000 7f0b73fff640 1 -- 172.21.15.70:0/1832668058 >> v2:172.21.15.70:3300/0 conn(0x7f0b7c06bc00 msgr2=0x7f0b7c06bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:08.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.081+0000 7f0b73fff640 1 --2- 172.21.15.70:0/1832668058 >> v2:172.21.15.70:3300/0 conn(0x7f0b7c06bc00 0x7f0b7c06bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.081+0000 7f0b73fff640 1 -- 172.21.15.70:0/1832668058 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c06d1f0 con 0x7f0b7c06c520 2024-09-13T15:44:08.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.081+0000 7f0b7affd640 1 --2- 172.21.15.70:0/1832668058 >> v2:172.21.15.70:3300/0 conn(0x7f0b7c06bc00 0x7f0b7c06bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:08.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.081+0000 7f0b7b7fe640 1 --2- 172.21.15.70:0/1832668058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b7c12d740 0x7f0b7c12dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:08.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.081+0000 7f0b73fff640 1 --2- 172.21.15.70:0/1832668058 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0b7c06c520 0x7f0b7c12d200 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009870 tx=0x7f0b6402a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c83fd29b0b3b8fb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:08.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.082+0000 7f0b7a7fc640 1 -- 172.21.15.70:0/1832668058 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6402adf0 con 0x7f0b7c06c520 2024-09-13T15:44:08.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.082+0000 7f0b7a7fc640 1 -- 172.21.15.70:0/1832668058 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b64032440 con 0x7f0b7c06c520 2024-09-13T15:44:08.085 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.082+0000 7f0b7a7fc640 1 -- 172.21.15.70:0/1832668058 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b640327d0 con 0x7f0b7c06c520 2024-09-13T15:44:08.087 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.082+0000 7f0b81353640 1 -- 172.21.15.70:0/1832668058 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0b7c06c520 msgr2=0x7f0b7c12d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:08.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.082+0000 7f0b81353640 1 --2- 172.21.15.70:0/1832668058 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0b7c06c520 0x7f0b7c12d200 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009870 tx=0x7f0b6402a0a0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.082+0000 7f0b81353640 1 -- 172.21.15.70:0/1832668058 shutdown_connections 2024-09-13T15:44:08.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.082+0000 7f0b81353640 1 --2- 172.21.15.70:0/1832668058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b7c12d740 0x7f0b7c12dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.082+0000 7f0b81353640 1 --2- 172.21.15.70:0/1832668058 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0b7c06c520 0x7f0b7c12d200 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.082+0000 7f0b81353640 1 --2- 172.21.15.70:0/1832668058 >> v2:172.21.15.70:3300/0 conn(0x7f0b7c06bc00 0x7f0b7c06bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.082+0000 7f0b81353640 1 -- 172.21.15.70:0/1832668058 >> 172.21.15.70:0/1832668058 conn(0x7f0b7c09fc40 msgr2=0x7f0b7c0a0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:08.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.082+0000 7f0b81353640 1 -- 172.21.15.70:0/1832668058 shutdown_connections 2024-09-13T15:44:08.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.082+0000 7f0b81353640 1 -- 172.21.15.70:0/1832668058 wait complete. 2024-09-13T15:44:08.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.083+0000 7f0b81353640 1 Processor -- start 2024-09-13T15:44:08.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.083+0000 7f0b81353640 1 -- start start 2024-09-13T15:44:08.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.083+0000 7f0b81353640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0b7c06bc00 0x7f0b7c132540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:08.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.083+0000 7f0b81353640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b7c06c520 0x7f0b7c132a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:08.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.083+0000 7f0b81353640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b7c12d740 0x7f0b7c132fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:08.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.083+0000 7f0b81353640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c130ed0 con 0x7f0b7c06c520 2024-09-13T15:44:08.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.083+0000 7f0b81353640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c1316d0 con 0x7f0b7c12d740 2024-09-13T15:44:08.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.083+0000 7f0b81353640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c131eb0 con 0x7f0b7c06bc00 2024-09-13T15:44:08.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.083+0000 7f0b7affd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0b7c06bc00 0x7f0b7c132540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:08.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.083+0000 7f0b7b7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b7c12d740 0x7f0b7c132fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:08.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.083+0000 7f0b7b7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b7c12d740 0x7f0b7c132fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33298/0 (socket says 172.21.15.70:33298) 2024-09-13T15:44:08.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.084+0000 7f0b7b7fe640 1 -- 172.21.15.70:0/919576178 learned_addr learned my addr 172.21.15.70:0/919576178 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:08.089 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.084+0000 7f0b73fff640 1 --2- 172.21.15.70:0/919576178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b7c06c520 0x7f0b7c132a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:08.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.084+0000 7f0b7b7fe640 1 -- 172.21.15.70:0/919576178 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0b7c06bc00 msgr2=0x7f0b7c132540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:08.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.084+0000 7f0b7b7fe640 1 --2- 172.21.15.70:0/919576178 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0b7c06bc00 0x7f0b7c132540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.084+0000 7f0b7b7fe640 1 -- 172.21.15.70:0/919576178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b7c06c520 msgr2=0x7f0b7c132a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:08.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.084+0000 7f0b7b7fe640 1 --2- 172.21.15.70:0/919576178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b7c06c520 0x7f0b7c132a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.084+0000 7f0b7b7fe640 1 -- 172.21.15.70:0/919576178 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c009400 con 0x7f0b7c12d740 2024-09-13T15:44:08.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.084+0000 7f0b7affd640 1 --2- 172.21.15.70:0/919576178 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0b7c06bc00 0x7f0b7c132540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:08.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.084+0000 7f0b73fff640 1 --2- 172.21.15.70:0/919576178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b7c06c520 0x7f0b7c132a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:08.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.084+0000 7f0b7b7fe640 1 --2- 172.21.15.70:0/919576178 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b7c12d740 0x7f0b7c132fc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009f00 tx=0x7f0b7400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:08.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.084+0000 7f0b737fe640 1 -- 172.21.15.70:0/919576178 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74009d70 con 0x7f0b7c12d740 2024-09-13T15:44:08.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.084+0000 7f0b737fe640 1 -- 172.21.15.70:0/919576178 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7400f040 con 0x7f0b7c12d740 2024-09-13T15:44:08.090 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.084+0000 7f0b81353640 1 -- 172.21.15.70:0/919576178 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b74009a70 con 0x7f0b7c12d740 2024-09-13T15:44:08.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.084+0000 7f0b737fe640 1 -- 172.21.15.70:0/919576178 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74003bb0 con 0x7f0b7c12d740 2024-09-13T15:44:08.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.084+0000 7f0b81353640 1 -- 172.21.15.70:0/919576178 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b640094d0 con 0x7f0b7c12d740 2024-09-13T15:44:08.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.085+0000 7f0b737fe640 1 -- 172.21.15.70:0/919576178 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f0b74014480 con 0x7f0b7c12d740 2024-09-13T15:44:08.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.085+0000 7f0b81353640 1 -- 172.21.15.70:0/919576178 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b7c12d200 con 0x7f0b7c12d740 2024-09-13T15:44:08.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.085+0000 7f0b737fe640 1 -- 172.21.15.70:0/919576178 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f0b7405d420 con 0x7f0b7c12d740 2024-09-13T15:44:08.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.089+0000 7f0b737fe640 1 -- 172.21.15.70:0/919576178 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f0b7403b030 con 0x7f0b7c12d740 2024-09-13T15:44:08.221 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.219+0000 7f0b81353640 1 -- 172.21.15.70:0/919576178 --> v1:172.21.15.70:6816/1229510400 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b7c06a6b0 con 0x7f0b5c047560 2024-09-13T15:44:08.222 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.221+0000 7f0b737fe640 1 -- 172.21.15.70:0/919576178 <== mgr.24114 v1:172.21.15.70:6816/1229510400 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34750 (unknown 2965378022 0 4254729499) 0x7f0b7c06a6b0 con 0x7f0b5c047560 2024-09-13T15:44:08.222 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:44:08.222 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-13T15:44:08.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.222+0000 7f0b81353640 1 -- 172.21.15.70:0/919576178 >> v1:172.21.15.70:6816/1229510400 conn(0x7f0b5c047560 legacy=0x7f0b5c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:08.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.222+0000 7f0b81353640 1 -- 172.21.15.70:0/919576178 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b7c12d740 msgr2=0x7f0b7c132fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:08.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.222+0000 7f0b81353640 1 --2- 172.21.15.70:0/919576178 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b7c12d740 0x7f0b7c132fc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009f00 tx=0x7f0b7400c490 comp rx=0 tx=0).stop 2024-09-13T15:44:08.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.222+0000 7f0b81353640 1 -- 172.21.15.70:0/919576178 shutdown_connections 2024-09-13T15:44:08.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.222+0000 7f0b81353640 1 --2- 172.21.15.70:0/919576178 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0b7c12d740 0x7f0b7c132fc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.222+0000 7f0b81353640 1 --2- 172.21.15.70:0/919576178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b7c06c520 0x7f0b7c132a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.224 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.222+0000 7f0b81353640 1 --2- 172.21.15.70:0/919576178 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0b7c06bc00 0x7f0b7c132540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.222+0000 7f0b81353640 1 -- 172.21.15.70:0/919576178 >> 172.21.15.70:0/919576178 conn(0x7f0b7c09fc40 msgr2=0x7f0b7c12c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:08.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.222+0000 7f0b81353640 1 -- 172.21.15.70:0/919576178 shutdown_connections 2024-09-13T15:44:08.225 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.222+0000 7f0b81353640 1 -- 172.21.15.70:0/919576178 wait complete. 2024-09-13T15:44:08.234 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-13T15:44:07.541008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":128996,"kb_used_data":2216,"kb_used_omap":17,"kb_used_meta":126702,"kb_avail":377358364,"statfs":{"total":386547056640,"available":386414964736,"internally_reserved":0,"allocated":2269184,"data_stored":1558473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":129743093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.945829"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599825+0000","last_change":"2024-09-13T15:43:56.791803+0000","last_active":"2024-09-13T15:44:01.599825+0000","last_peered":"2024-09-13T15:44:01.599825+0000","last_clean":"2024-09-13T15:44:01.599825+0000","last_became_active":"2024-09-13T15:43:56.791454+0000","last_became_peered":"2024-09-13T15:43:56.791454+0000","last_unstale":"2024-09-13T15:44:01.599825+0000","last_undegraded":"2024-09-13T15:44:01.599825+0000","last_fullsized":"2024-09-13T15:44:01.599825+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:38.078282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035985100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599965+0000","last_change":"2024-09-13T15:43:54.574295+0000","last_active":"2024-09-13T15:44:01.599965+0000","last_peered":"2024-09-13T15:44:01.599965+0000","last_clean":"2024-09-13T15:44:01.599965+0000","last_became_active":"2024-09-13T15:43:54.573978+0000","last_became_peered":"2024-09-13T15:43:54.573978+0000","last_unstale":"2024-09-13T15:44:01.599965+0000","last_undegraded":"2024-09-13T15:44:01.599965+0000","last_fullsized":"2024-09-13T15:44:01.599965+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:09.974326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054553999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599736+0000","last_change":"2024-09-13T15:43:56.792140+0000","last_active":"2024-09-13T15:44:01.599736+0000","last_peered":"2024-09-13T15:44:01.599736+0000","last_clean":"2024-09-13T15:44:01.599736+0000","last_became_active":"2024-09-13T15:43:56.791789+0000","last_became_peered":"2024-09-13T15:43:56.791789+0000","last_unstale":"2024-09-13T15:44:01.599736+0000","last_undegraded":"2024-09-13T15:44:01.599736+0000","last_fullsized":"2024-09-13T15:44:01.599736+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:51.347592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011766770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599970+0000","last_change":"2024-09-13T15:43:54.573858+0000","last_active":"2024-09-13T15:44:01.599970+0000","last_peered":"2024-09-13T15:44:01.599970+0000","last_clean":"2024-09-13T15:44:01.599970+0000","last_became_active":"2024-09-13T15:43:54.573540+0000","last_became_peered":"2024-09-13T15:43:54.573540+0000","last_unstale":"2024-09-13T15:44:01.599970+0000","last_undegraded":"2024-09-13T15:44:01.599970+0000","last_fullsized":"2024-09-13T15:44:01.599970+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:50.925568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024452499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599730+0000","last_change":"2024-09-13T15:43:56.584865+0000","last_active":"2024-09-13T15:44:01.599730+0000","last_peered":"2024-09-13T15:44:01.599730+0000","last_clean":"2024-09-13T15:44:01.599730+0000","last_became_active":"2024-09-13T15:43:56.584517+0000","last_became_peered":"2024-09-13T15:43:56.584517+0000","last_unstale":"2024-09-13T15:44:01.599730+0000","last_undegraded":"2024-09-13T15:44:01.599730+0000","last_fullsized":"2024-09-13T15:44:01.599730+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:29:26.245037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094542300000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T15:43:56.583705+0000","last_change":"2024-09-13T15:43:56.582866+0000","last_active":"2024-09-13T15:43:56.583705+0000","last_peered":"2024-09-13T15:43:56.583705+0000","last_clean":"2024-09-13T15:43:56.583705+0000","last_became_active":"2024-09-13T15:43:56.582550+0000","last_became_peered":"2024-09-13T15:43:56.582550+0000","last_unstale":"2024-09-13T15:43:56.583705+0000","last_undegraded":"2024-09-13T15:43:56.583705+0000","last_fullsized":"2024-09-13T15:43:56.583705+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:26.074766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089033199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599643+0000","last_change":"2024-09-13T15:43:56.585264+0000","last_active":"2024-09-13T15:44:01.599643+0000","last_peered":"2024-09-13T15:44:01.599643+0000","last_clean":"2024-09-13T15:44:01.599643+0000","last_became_active":"2024-09-13T15:43:56.584633+0000","last_became_peered":"2024-09-13T15:43:56.584633+0000","last_unstale":"2024-09-13T15:44:01.599643+0000","last_undegraded":"2024-09-13T15:44:01.599643+0000","last_fullsized":"2024-09-13T15:44:01.599643+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:29.252314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034895899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599737+0000","last_change":"2024-09-13T15:43:56.583583+0000","last_active":"2024-09-13T15:44:01.599737+0000","last_peered":"2024-09-13T15:44:01.599737+0000","last_clean":"2024-09-13T15:44:01.599737+0000","last_became_active":"2024-09-13T15:43:56.582549+0000","last_became_peered":"2024-09-13T15:43:56.582549+0000","last_unstale":"2024-09-13T15:44:01.599737+0000","last_undegraded":"2024-09-13T15:44:01.599737+0000","last_fullsized":"2024-09-13T15:44:01.599737+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:58:03.411133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276560000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'63","reported_seq":140,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599581+0000","last_change":"2024-09-13T15:43:54.574397+0000","last_active":"2024-09-13T15:44:01.599581+0000","last_peered":"2024-09-13T15:44:01.599581+0000","last_clean":"2024-09-13T15:44:01.599581+0000","last_became_active":"2024-09-13T15:43:54.574044+0000","last_became_peered":"2024-09-13T15:43:54.574044+0000","last_unstale":"2024-09-13T15:44:01.599581+0000","last_undegraded":"2024-09-13T15:44:01.599581+0000","last_fullsized":"2024-09-13T15:44:01.599581+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_clean_scrub_stamp":"2024-09-13T15:42:21.689345+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:48:10.619565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":28,"seq":120259084292,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":268,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":274432,"data_stored":95335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.215}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]}]},{"osd":2,"up_from":27,"seq":115964116996,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":256,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":262144,"data_stored":91751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]}]},{"osd":1,"up_from":26,"seq":111669149700,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":868352,"data_stored":685703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":32435774},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]}]},{"osd":0,"up_from":25,"seq":107374182405,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":844,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":864256,"data_stored":685684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.245}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T15:44:08.235 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T15:44:08.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.330+0000 7fc7d3427640 1 Processor -- start 2024-09-13T15:44:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7d3427640 1 -- start start 2024-09-13T15:44:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7d3427640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7cc06c600 0x7fc7cc132710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7d3427640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc7cc132c50 0x7fc7cc135040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7d3427640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7cc097940 con 0x7fc7cc06bb10 2024-09-13T15:44:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7d3427640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7cc097500 con 0x7fc7cc132c50 2024-09-13T15:44:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7d3427640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc06d100 con 0x7fc7cc06c600 2024-09-13T15:44:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7d19c4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc7cc132c50 0x7fc7cc135040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7d09c2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7cc06c600 0x7fc7cc132710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7d11c3640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc7cc06bb10 0x7fc7cc06bef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55460/0 (socket says 172.21.15.70:55460) 2024-09-13T15:44:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7d11c3640 1 -- 172.21.15.70:0/1327985841 learned_addr learned my addr 172.21.15.70:0/1327985841 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7d19c4640 1 -- 172.21.15.70:0/1327985841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7cc06c600 msgr2=0x7fc7cc132710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7d19c4640 1 --2- 172.21.15.70:0/1327985841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7cc06c600 0x7fc7cc132710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7d19c4640 1 -- 172.21.15.70:0/1327985841 >> v1:172.21.15.70:6789/0 conn(0x7fc7cc06bb10 legacy=0x7fc7cc06bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:08.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7bffff640 1 -- 172.21.15.70:0/1327985841 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527144634 0 0) 0x7fc7cc097940 con 0x7fc7cc06bb10 2024-09-13T15:44:08.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7d19c4640 1 -- 172.21.15.70:0/1327985841 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc06da40 con 0x7fc7cc132c50 2024-09-13T15:44:08.334 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.331+0000 7fc7d09c2640 1 --2- 172.21.15.70:0/1327985841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7cc06c600 0x7fc7cc132710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:08.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.332+0000 7fc7d19c4640 1 --2- 172.21.15.70:0/1327985841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc132c50 0x7fc7cc135040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009d90 tx=0x7fc7c402a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6303d98999c5f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:08.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.332+0000 7fc7bffff640 1 -- 172.21.15.70:0/1327985841 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4007740 con 0x7fc7cc132c50 2024-09-13T15:44:08.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.332+0000 7fc7bffff640 1 -- 172.21.15.70:0/1327985841 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c402d050 con 0x7fc7cc132c50 2024-09-13T15:44:08.336 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.332+0000 7fc7bffff640 1 -- 172.21.15.70:0/1327985841 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4007ac0 con 0x7fc7cc132c50 2024-09-13T15:44:08.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.333+0000 7fc7d3427640 1 -- 172.21.15.70:0/1327985841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc132c50 msgr2=0x7fc7cc135040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:08.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.333+0000 7fc7d3427640 1 --2- 172.21.15.70:0/1327985841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc132c50 0x7fc7cc135040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009d90 tx=0x7fc7c402a530 comp rx=0 tx=0).stop 2024-09-13T15:44:08.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.333+0000 7fc7d3427640 1 -- 172.21.15.70:0/1327985841 shutdown_connections 2024-09-13T15:44:08.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.333+0000 7fc7d3427640 1 --2- 172.21.15.70:0/1327985841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc132c50 0x7fc7cc135040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.333+0000 7fc7d3427640 1 --2- 172.21.15.70:0/1327985841 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7cc06c600 0x7fc7cc132710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.333+0000 7fc7d3427640 1 -- 172.21.15.70:0/1327985841 >> 172.21.15.70:0/1327985841 conn(0x7fc7cc09fe20 msgr2=0x7fc7cc0a0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:08.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.333+0000 7fc7d3427640 1 -- 172.21.15.70:0/1327985841 shutdown_connections 2024-09-13T15:44:08.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.333+0000 7fc7d3427640 1 -- 172.21.15.70:0/1327985841 wait complete. 2024-09-13T15:44:08.337 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d3427640 1 Processor -- start 2024-09-13T15:44:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d3427640 1 -- start start 2024-09-13T15:44:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d3427640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc06bb10 0x7fc7cc1bec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d3427640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc7cc06c600 0x7fc7cc1bf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d3427640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7cc132c50 0x7fc7cc1c1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d3427640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc12cb20 con 0x7fc7cc06bb10 2024-09-13T15:44:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d3427640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc127d30 con 0x7fc7cc132c50 2024-09-13T15:44:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d3427640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc12ecf0 con 0x7fc7cc06c600 2024-09-13T15:44:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d11c3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc06bb10 0x7fc7cc1bec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d19c4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7cc132c50 0x7fc7cc1c1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d09c2640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc7cc06c600 0x7fc7cc1bf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d11c3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc06bb10 0x7fc7cc1bec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54206/0 (socket says 172.21.15.70:54206) 2024-09-13T15:44:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d11c3640 1 -- 172.21.15.70:0/1081562630 learned_addr learned my addr 172.21.15.70:0/1081562630 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d19c4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7cc132c50 0x7fc7cc1c1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33330/0 (socket says 172.21.15.70:33330) 2024-09-13T15:44:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d11c3640 1 -- 172.21.15.70:0/1081562630 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc7cc06c600 msgr2=0x7fc7cc1bf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d11c3640 1 --2- 172.21.15.70:0/1081562630 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc7cc06c600 0x7fc7cc1bf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d11c3640 1 -- 172.21.15.70:0/1081562630 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7cc132c50 msgr2=0x7fc7cc1c1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d11c3640 1 --2- 172.21.15.70:0/1081562630 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7cc132c50 0x7fc7cc1c1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.334+0000 7fc7d11c3640 1 -- 172.21.15.70:0/1081562630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c8009400 con 0x7fc7cc06bb10 2024-09-13T15:44:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.335+0000 7fc7d11c3640 1 --2- 172.21.15.70:0/1081562630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc06bb10 0x7fc7cc1bec40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c4a0 tx=0x7fc7c000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.335+0000 7fc7bdffb640 1 -- 172.21.15.70:0/1081562630 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c00074e0 con 0x7fc7cc06bb10 2024-09-13T15:44:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.335+0000 7fc7bdffb640 1 -- 172.21.15.70:0/1081562630 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c0007640 con 0x7fc7cc06bb10 2024-09-13T15:44:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.335+0000 7fc7d3427640 1 -- 172.21.15.70:0/1081562630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c4009a70 con 0x7fc7cc06bb10 2024-09-13T15:44:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.335+0000 7fc7bdffb640 1 -- 172.21.15.70:0/1081562630 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0004a00 con 0x7fc7cc06bb10 2024-09-13T15:44:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.335+0000 7fc7d3427640 1 -- 172.21.15.70:0/1081562630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7cc1bf6c0 con 0x7fc7cc06bb10 2024-09-13T15:44:08.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.336+0000 7fc7bdffb640 1 -- 172.21.15.70:0/1081562630 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fc7c0004b60 con 0x7fc7cc06bb10 2024-09-13T15:44:08.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.336+0000 7fc7d3427640 1 -- 172.21.15.70:0/1081562630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7cc06bef0 con 0x7fc7cc06bb10 2024-09-13T15:44:08.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.336+0000 7fc7bdffb640 1 -- 172.21.15.70:0/1081562630 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fc7c005be10 con 0x7fc7cc06bb10 2024-09-13T15:44:08.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.340+0000 7fc7bdffb640 1 -- 172.21.15.70:0/1081562630 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fc7c0039a80 con 0x7fc7cc06bb10 2024-09-13T15:44:08.476 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.475+0000 7fc7d3427640 1 -- 172.21.15.70:0/1081562630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7cc06a6b0 con 0x7fc7cc06bb10 2024-09-13T15:44:08.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.476+0000 7fc7bdffb640 1 -- 172.21.15.70:0/1081562630 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+9049 (secure 0 0 0) 0x7fc7c0018020 con 0x7fc7cc06bb10 2024-09-13T15:44:08.477 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:44:08.477 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":32,"fsid":"45bbcdad-f4fb-4639-ab42-ccda8de151dd","created":"2024-09-13T15:42:09.863982+0000","modified":"2024-09-13T15:44:02.597838+0000","last_up_change":"2024-09-13T15:43:55.573914+0000","last_in_change":"2024-09-13T15:42:11.714506+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"quincy","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T15:42:21.000499+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T15:42:26.149909+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"265c6f1d-08a7-4c43-b2a6-2417ab814825","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":19,"up_from":25,"up_thru":25,"down_at":20,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6800","nonce":2246103037}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6801","nonce":2246103037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6803","nonce":2246103037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6802","nonce":2246103037}]},"public_addr":"172.21.15.70:6800/2246103037","cluster_addr":"172.21.15.70:6801/2246103037","heartbeat_back_addr":"172.21.15.70:6803/2246103037","heartbeat_front_addr":"172.21.15.70:6802/2246103037","state":["exists","up"]},{"osd":1,"uuid":"ba4d90f9-6972-45df-800f-92ad808b06ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":20,"up_from":26,"up_thru":26,"down_at":21,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6804","nonce":853418053}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6805","nonce":853418053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6807","nonce":853418053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6806","nonce":853418053}]},"public_addr":"172.21.15.70:6804/853418053","cluster_addr":"172.21.15.70:6805/853418053","heartbeat_back_addr":"172.21.15.70:6807/853418053","heartbeat_front_addr":"172.21.15.70:6806/853418053","state":["exists","up"]},{"osd":2,"uuid":"b74d8eb5-3082-4d69-a755-7f5f92dec5b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":27,"up_thru":28,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6808","nonce":2115999109}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6809","nonce":2115999109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6811","nonce":2115999109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6810","nonce":2115999109}]},"public_addr":"172.21.15.70:6808/2115999109","cluster_addr":"172.21.15.70:6809/2115999109","heartbeat_back_addr":"172.21.15.70:6811/2115999109","heartbeat_front_addr":"172.21.15.70:6810/2115999109","state":["exists","up"]},{"osd":3,"uuid":"02835b7d-e2cf-4ef8-87ba-b225593689d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":28,"up_thru":28,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6812","nonce":2095410362}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6813","nonce":2095410362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6815","nonce":2095410362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.70:6814","nonce":2095410362}]},"public_addr":"172.21.15.70:6812/2095410362","cluster_addr":"172.21.15.70:6813/2095410362","heartbeat_back_addr":"172.21.15.70:6815/2095410362","heartbeat_front_addr":"172.21.15.70:6814/2095410362","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.853420+0000","dead_epoch":20},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.807491+0000","dead_epoch":21},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.925567+0000","dead_epoch":22},{"osd":3,"down_stamp":"2024-09-13T15:43:48.549448+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T15:42:16.883852+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6816/32242":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/2914106548":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3671770748":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3345804336":"2024-09-14T15:43:41.526957+0000","172.21.15.70:0/3873928986":"2024-09-14T15:43:41.526957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T15:44:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.478+0000 7fc7d3427640 1 -- 172.21.15.70:0/1081562630 >> v1:172.21.15.70:6816/1229510400 conn(0x7fc7b0047560 legacy=0x7fc7b0049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.478+0000 7fc7d3427640 1 -- 172.21.15.70:0/1081562630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc06bb10 msgr2=0x7fc7cc1bec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.478+0000 7fc7d3427640 1 --2- 172.21.15.70:0/1081562630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc06bb10 0x7fc7cc1bec40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c4a0 tx=0x7fc7c000c960 comp rx=0 tx=0).stop 2024-09-13T15:44:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.478+0000 7fc7d3427640 1 -- 172.21.15.70:0/1081562630 shutdown_connections 2024-09-13T15:44:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.478+0000 7fc7d3427640 1 --2- 172.21.15.70:0/1081562630 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc7cc132c50 0x7fc7cc1c1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.478+0000 7fc7d3427640 1 --2- 172.21.15.70:0/1081562630 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fc7cc06c600 0x7fc7cc1bf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.478+0000 7fc7d3427640 1 --2- 172.21.15.70:0/1081562630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7cc06bb10 0x7fc7cc1bec40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:08.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.478+0000 7fc7d3427640 1 -- 172.21.15.70:0/1081562630 >> 172.21.15.70:0/1081562630 conn(0x7fc7cc09fe20 msgr2=0x7fc7cc09bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:08.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.478+0000 7fc7d3427640 1 -- 172.21.15.70:0/1081562630 shutdown_connections 2024-09-13T15:44:08.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:08.478+0000 7fc7d3427640 1 -- 172.21.15.70:0/1081562630 wait complete. 2024-09-13T15:44:09.490 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T15:44:09.491 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T15:44:09.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.595+0000 7f3ea5c3a640 1 Processor -- start 2024-09-13T15:44:09.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.595+0000 7f3ea5c3a640 1 -- start start 2024-09-13T15:44:09.597 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.595+0000 7f3ea5c3a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ea006b970 0x7f3ea006bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.595+0000 7f3ea5c3a640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ea006c290 0x7f3ea00957a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.595+0000 7f3ea5c3a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea0097d40 con 0x7f3ea006eef0 2024-09-13T15:44:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.595+0000 7f3ea5c3a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00702d0 con 0x7f3ea006b970 2024-09-13T15:44:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.595+0000 7f3ea5c3a640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00705e0 con 0x7f3ea006c290 2024-09-13T15:44:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.596+0000 7f3e9f7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ea006b970 0x7f3ea006bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.596+0000 7f3e9ffff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ea006c290 0x7f3ea00957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.596+0000 7f3e9effd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3ea006eef0 0x7f3ea0095090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55474/0 (socket says 172.21.15.70:55474) 2024-09-13T15:44:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.596+0000 7f3e9effd640 1 -- 172.21.15.70:0/465304530 learned_addr learned my addr 172.21.15.70:0/465304530 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.596+0000 7f3e9e7fc640 1 -- 172.21.15.70:0/465304530 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4018840282 0 0) 0x7f3ea0097d40 con 0x7f3ea006eef0 2024-09-13T15:44:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.596+0000 7f3e9e7fc640 1 -- 172.21.15.70:0/465304530 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e88003520 con 0x7f3ea006eef0 2024-09-13T15:44:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.596+0000 7f3e9ffff640 1 -- 172.21.15.70:0/465304530 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ea006b970 msgr2=0x7f3ea006bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.596+0000 7f3e9ffff640 1 --2- 172.21.15.70:0/465304530 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ea006b970 0x7f3ea006bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.596+0000 7f3e9ffff640 1 -- 172.21.15.70:0/465304530 >> v1:172.21.15.70:6789/0 conn(0x7f3ea006eef0 legacy=0x7f3ea0095090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.596+0000 7f3e9ffff640 1 -- 172.21.15.70:0/465304530 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea006c6c0 con 0x7f3ea006c290 2024-09-13T15:44:09.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.596+0000 7f3e9f7fe640 1 --2- 172.21.15.70:0/465304530 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ea006b970 0x7f3ea006bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:09.601 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.597+0000 7f3e9ffff640 1 --2- 172.21.15.70:0/465304530 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ea006c290 0x7f3ea00957a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3e980097d0 tx=0x7f3e9802a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e2aa0a4c888a85b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.597+0000 7f3e9e7fc640 1 -- 172.21.15.70:0/465304530 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9802ad90 con 0x7f3ea006c290 2024-09-13T15:44:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.597+0000 7f3e9e7fc640 1 -- 172.21.15.70:0/465304530 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e98032440 con 0x7f3ea006c290 2024-09-13T15:44:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.597+0000 7f3e9e7fc640 1 -- 172.21.15.70:0/465304530 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e980327d0 con 0x7f3ea006c290 2024-09-13T15:44:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.597+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/465304530 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ea006c290 msgr2=0x7f3ea00957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.597+0000 7f3ea5c3a640 1 --2- 172.21.15.70:0/465304530 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ea006c290 0x7f3ea00957a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3e980097d0 tx=0x7f3e9802a040 comp rx=0 tx=0).stop 2024-09-13T15:44:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.597+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/465304530 shutdown_connections 2024-09-13T15:44:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.597+0000 7f3ea5c3a640 1 --2- 172.21.15.70:0/465304530 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ea006c290 0x7f3ea00957a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.598+0000 7f3ea5c3a640 1 --2- 172.21.15.70:0/465304530 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ea006b970 0x7f3ea006bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.598+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/465304530 >> 172.21.15.70:0/465304530 conn(0x7f3ea0098ab0 msgr2=0x7f3ea00963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.598+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/465304530 shutdown_connections 2024-09-13T15:44:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.598+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/465304530 wait complete. 2024-09-13T15:44:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.598+0000 7f3ea5c3a640 1 Processor -- start 2024-09-13T15:44:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3ea5c3a640 1 -- start start 2024-09-13T15:44:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3ea5c3a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea006b970 0x7f3ea006ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3ea5c3a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ea006c290 0x7f3ea006cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3ea5c3a640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ea006eef0 0x7f3ea006d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3ea5c3a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01251b0 con 0x7f3ea006b970 2024-09-13T15:44:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3ea5c3a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea012c8e0 con 0x7f3ea006c290 2024-09-13T15:44:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3ea5c3a640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea0095740 con 0x7f3ea006eef0 2024-09-13T15:44:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3e9effd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ea006c290 0x7f3ea006cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3e9f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea006b970 0x7f3ea006ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3e9effd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ea006c290 0x7f3ea006cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33344/0 (socket says 172.21.15.70:33344) 2024-09-13T15:44:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3e9ffff640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ea006eef0 0x7f3ea006d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3e9effd640 1 -- 172.21.15.70:0/3880211735 learned_addr learned my addr 172.21.15.70:0/3880211735 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3e9f7fe640 1 -- 172.21.15.70:0/3880211735 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ea006eef0 msgr2=0x7f3ea006d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3e9f7fe640 1 --2- 172.21.15.70:0/3880211735 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f3ea006eef0 0x7f3ea006d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3e9f7fe640 1 -- 172.21.15.70:0/3880211735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ea006c290 msgr2=0x7f3ea006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3e9f7fe640 1 --2- 172.21.15.70:0/3880211735 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3ea006c290 0x7f3ea006cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3e9f7fe640 1 -- 172.21.15.70:0/3880211735 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e90009a70 con 0x7f3ea006b970 2024-09-13T15:44:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.599+0000 7f3e9f7fe640 1 --2- 172.21.15.70:0/3880211735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea006b970 0x7f3ea006ca80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000efc0 tx=0x7f3e9000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.600+0000 7f3e87fff640 1 -- 172.21.15.70:0/3880211735 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90014680 con 0x7f3ea006b970 2024-09-13T15:44:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.600+0000 7f3e87fff640 1 -- 172.21.15.70:0/3880211735 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9000f040 con 0x7f3ea006b970 2024-09-13T15:44:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.600+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/3880211735 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e98009400 con 0x7f3ea006b970 2024-09-13T15:44:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.600+0000 7f3e87fff640 1 -- 172.21.15.70:0/3880211735 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90014a20 con 0x7f3ea006b970 2024-09-13T15:44:09.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.600+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/3880211735 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea006e3a0 con 0x7f3ea006b970 2024-09-13T15:44:09.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.600+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/3880211735 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3ea009b790 con 0x7f3ea006b970 2024-09-13T15:44:09.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.601+0000 7f3e87fff640 1 -- 172.21.15.70:0/3880211735 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f3e90014b80 con 0x7f3ea006b970 2024-09-13T15:44:09.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.601+0000 7f3e87fff640 1 -- 172.21.15.70:0/3880211735 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f3e9005d800 con 0x7f3ea006b970 2024-09-13T15:44:09.605 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.601+0000 7f3e87fff640 1 -- 172.21.15.70:0/3880211735 --> v1:172.21.15.70:6800/2246103037 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e8c051aa0 con 0x7f3e8c04ddf0 2024-09-13T15:44:09.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.601+0000 7f3e87fff640 1 -- 172.21.15.70:0/3880211735 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e900622c0 con 0x7f3ea006b970 2024-09-13T15:44:09.606 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.602+0000 7f3e87fff640 1 -- 172.21.15.70:0/3880211735 <== osd.0 v1:172.21.15.70:6800/2246103037 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7f3e8c051aa0 con 0x7f3e8c04ddf0 2024-09-13T15:44:09.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.614+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/3880211735 --> v1:172.21.15.70:6800/2246103037 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3ea01337a0 con 0x7f3e8c04ddf0 2024-09-13T15:44:09.615 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.614+0000 7f3e87fff640 1 -- 172.21.15.70:0/3880211735 <== osd.0 v1:172.21.15.70:6800/2246103037 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (unknown 0 0 2175974996) 0x7f3e9802ab90 con 0x7f3e8c04ddf0 2024-09-13T15:44:09.616 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-13T15:44:09.616 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-13T15:44:09.616 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-13T15:44:09.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.614+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/3880211735 >> v1:172.21.15.70:6800/2246103037 conn(0x7f3e8c04ddf0 legacy=0x7f3e8c050230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.614+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/3880211735 >> v1:172.21.15.70:6816/1229510400 conn(0x7f3e8c047560 legacy=0x7f3e8c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.614+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/3880211735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea006b970 msgr2=0x7f3ea006ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.614+0000 7f3ea5c3a640 1 --2- 172.21.15.70:0/3880211735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea006b970 0x7f3ea006ca80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000efc0 tx=0x7f3e9000c490 comp rx=0 tx=0).stop 2024-09-13T15:44:09.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.614+0000 7f3e9ffff640 1 -- 172.21.15.70:0/3880211735 reap_dead start 2024-09-13T15:44:09.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.614+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/3880211735 shutdown_connections 2024-09-13T15:44:09.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.614+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/3880211735 >> 172.21.15.70:0/3880211735 conn(0x7f3ea0098ab0 msgr2=0x7f3ea0099890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:09.617 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.615+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/3880211735 shutdown_connections 2024-09-13T15:44:09.618 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.615+0000 7f3ea5c3a640 1 -- 172.21.15.70:0/3880211735 wait complete. 2024-09-13T15:44:09.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T15:44:09.724 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.723+0000 7f0d5bc4f640 1 Processor -- start 2024-09-13T15:44:09.725 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.723+0000 7f0d5bc4f640 1 -- start start 2024-09-13T15:44:09.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.724+0000 7f0d5bc4f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0d5412df50 0x7f0d54132650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:09.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.724+0000 7f0d5bc4f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d54132b90 0x7f0d54134f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:09.726 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.724+0000 7f0d5bc4f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d5406d5d0 con 0x7f0d5412d330 2024-09-13T15:44:09.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.724+0000 7f0d5bc4f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d540971b0 con 0x7f0d5412df50 2024-09-13T15:44:09.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.724+0000 7f0d5bc4f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d540979f0 con 0x7f0d54132b90 2024-09-13T15:44:09.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.724+0000 7f0d591ea640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0d5412df50 0x7f0d54132650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:09.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.724+0000 7f0d599eb640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0d5412d330 0x7f0d5412d840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55490/0 (socket says 172.21.15.70:55490) 2024-09-13T15:44:09.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.724+0000 7f0d599eb640 1 -- 172.21.15.70:0/877219396 learned_addr learned my addr 172.21.15.70:0/877219396 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:09.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.724+0000 7f0d5a1ec640 1 --2- 172.21.15.70:0/877219396 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d54132b90 0x7f0d54134f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:09.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.724+0000 7f0d5a1ec640 1 -- 172.21.15.70:0/877219396 >> v2:172.21.15.70:3300/0 conn(0x7f0d5412df50 msgr2=0x7f0d54132650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:09.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.724+0000 7f0d5a1ec640 1 --2- 172.21.15.70:0/877219396 >> v2:172.21.15.70:3300/0 conn(0x7f0d5412df50 0x7f0d54132650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.727 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.724+0000 7f0d5a1ec640 1 -- 172.21.15.70:0/877219396 >> v1:172.21.15.70:6789/0 conn(0x7f0d5412d330 legacy=0x7f0d5412d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.724+0000 7f0d5a1ec640 1 -- 172.21.15.70:0/877219396 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d54097620 con 0x7f0d54132b90 2024-09-13T15:44:09.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.724+0000 7f0d589e9640 1 -- 172.21.15.70:0/877219396 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051688859 0 0) 0x7f0d5406d5d0 con 0x7f0d5412d330 2024-09-13T15:44:09.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.724+0000 7f0d591ea640 1 --2- 172.21.15.70:0/877219396 >> v2:172.21.15.70:3300/0 conn(0x7f0d5412df50 0x7f0d54132650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:09.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.725+0000 7f0d5a1ec640 1 --2- 172.21.15.70:0/877219396 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d54132b90 0x7f0d54134f80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009d90 tx=0x7f0d4c02a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af7472c4ce66bd1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:09.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.725+0000 7f0d589e9640 1 -- 172.21.15.70:0/877219396 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c007740 con 0x7f0d54132b90 2024-09-13T15:44:09.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.725+0000 7f0d589e9640 1 -- 172.21.15.70:0/877219396 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4c02d050 con 0x7f0d54132b90 2024-09-13T15:44:09.728 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.725+0000 7f0d589e9640 1 -- 172.21.15.70:0/877219396 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c007ac0 con 0x7f0d54132b90 2024-09-13T15:44:09.732 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.726+0000 7f0d5bc4f640 1 -- 172.21.15.70:0/877219396 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d54132b90 msgr2=0x7f0d54134f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.726+0000 7f0d5bc4f640 1 --2- 172.21.15.70:0/877219396 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d54132b90 0x7f0d54134f80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009d90 tx=0x7f0d4c02a530 comp rx=0 tx=0).stop 2024-09-13T15:44:09.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.726+0000 7f0d5bc4f640 1 -- 172.21.15.70:0/877219396 shutdown_connections 2024-09-13T15:44:09.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.726+0000 7f0d5bc4f640 1 --2- 172.21.15.70:0/877219396 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d54132b90 0x7f0d54134f80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.726+0000 7f0d5bc4f640 1 --2- 172.21.15.70:0/877219396 >> v2:172.21.15.70:3300/0 conn(0x7f0d5412df50 0x7f0d54132650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.726+0000 7f0d5bc4f640 1 -- 172.21.15.70:0/877219396 >> 172.21.15.70:0/877219396 conn(0x7f0d540a0060 msgr2=0x7f0d540a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:09.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.726+0000 7f0d5bc4f640 1 -- 172.21.15.70:0/877219396 shutdown_connections 2024-09-13T15:44:09.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.726+0000 7f0d5bc4f640 1 -- 172.21.15.70:0/877219396 wait complete. 2024-09-13T15:44:09.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d5bc4f640 1 Processor -- start 2024-09-13T15:44:09.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d5bc4f640 1 -- start start 2024-09-13T15:44:09.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d5bc4f640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0d5412d330 0x7f0d54124d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:09.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d5bc4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d5412df50 0x7f0d54125240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:09.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d5bc4f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d54132b90 0x7f0d54125780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:09.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d5bc4f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d54129f50 con 0x7f0d5412df50 2024-09-13T15:44:09.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d5bc4f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d541267e0 con 0x7f0d54132b90 2024-09-13T15:44:09.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d5bc4f640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5412ad00 con 0x7f0d5412d330 2024-09-13T15:44:09.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d599eb640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0d5412d330 0x7f0d54124d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:09.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d5a1ec640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d54132b90 0x7f0d54125780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:09.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d591ea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d5412df50 0x7f0d54125240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:09.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d599eb640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0d5412d330 0x7f0d54124d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:39776/0 (socket says 172.21.15.70:39776) 2024-09-13T15:44:09.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d591ea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d5412df50 0x7f0d54125240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54232/0 (socket says 172.21.15.70:54232) 2024-09-13T15:44:09.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d5a1ec640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d54132b90 0x7f0d54125780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33350/0 (socket says 172.21.15.70:33350) 2024-09-13T15:44:09.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d599eb640 1 -- 172.21.15.70:0/3804992998 learned_addr learned my addr 172.21.15.70:0/3804992998 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:09.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d599eb640 1 -- 172.21.15.70:0/3804992998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d54132b90 msgr2=0x7f0d54125780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d599eb640 1 --2- 172.21.15.70:0/3804992998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d54132b90 0x7f0d54125780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d599eb640 1 -- 172.21.15.70:0/3804992998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d5412df50 msgr2=0x7f0d54125240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d599eb640 1 --2- 172.21.15.70:0/3804992998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d5412df50 0x7f0d54125240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.727+0000 7f0d599eb640 1 -- 172.21.15.70:0/3804992998 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d44009400 con 0x7f0d5412d330 2024-09-13T15:44:09.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.728+0000 7f0d599eb640 1 --2- 172.21.15.70:0/3804992998 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0d5412d330 0x7f0d54124d00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000c4a0 tx=0x7f0d5000c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:09.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.728+0000 7f0d4a7fc640 1 -- 172.21.15.70:0/3804992998 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d500074e0 con 0x7f0d5412d330 2024-09-13T15:44:09.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.728+0000 7f0d4a7fc640 1 -- 172.21.15.70:0/3804992998 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d50007640 con 0x7f0d5412d330 2024-09-13T15:44:09.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.728+0000 7f0d5bc4f640 1 -- 172.21.15.70:0/3804992998 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4c009a70 con 0x7f0d5412d330 2024-09-13T15:44:09.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.728+0000 7f0d4a7fc640 1 -- 172.21.15.70:0/3804992998 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50004a00 con 0x7f0d5412d330 2024-09-13T15:44:09.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.728+0000 7f0d5bc4f640 1 -- 172.21.15.70:0/3804992998 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d54127170 con 0x7f0d5412d330 2024-09-13T15:44:09.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.729+0000 7f0d5bc4f640 1 -- 172.21.15.70:0/3804992998 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5412d840 con 0x7f0d5412d330 2024-09-13T15:44:09.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.729+0000 7f0d4a7fc640 1 -- 172.21.15.70:0/3804992998 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f0d50004b60 con 0x7f0d5412d330 2024-09-13T15:44:09.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.729+0000 7f0d4a7fc640 1 -- 172.21.15.70:0/3804992998 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f0d5005bd70 con 0x7f0d5412d330 2024-09-13T15:44:09.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.733+0000 7f0d4a7fc640 1 -- 172.21.15.70:0/3804992998 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f0d500399e0 con 0x7f0d5412d330 2024-09-13T15:44:09.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.870+0000 7f0d5bc4f640 1 -- 172.21.15.70:0/3804992998 --> v1:172.21.15.70:6816/1229510400 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d5406a6b0 con 0x7f0d38047590 2024-09-13T15:44:09.871 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.870+0000 7f0d4a7fc640 1 -- 172.21.15.70:0/3804992998 <== mgr.24114 v1:172.21.15.70:6816/1229510400 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (unknown 2057319476 0 0) 0x7f0d5406a6b0 con 0x7f0d38047590 2024-09-13T15:44:09.871 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T15:44:09.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.872+0000 7f0d5bc4f640 1 -- 172.21.15.70:0/3804992998 >> v1:172.21.15.70:6816/1229510400 conn(0x7f0d38047590 legacy=0x7f0d38049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.872+0000 7f0d5bc4f640 1 -- 172.21.15.70:0/3804992998 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0d5412d330 msgr2=0x7f0d54124d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.872+0000 7f0d5bc4f640 1 --2- 172.21.15.70:0/3804992998 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0d5412d330 0x7f0d54124d00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000c4a0 tx=0x7f0d5000c960 comp rx=0 tx=0).stop 2024-09-13T15:44:09.874 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.872+0000 7f0d5bc4f640 1 -- 172.21.15.70:0/3804992998 shutdown_connections 2024-09-13T15:44:09.874 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.872+0000 7f0d5bc4f640 1 --2- 172.21.15.70:0/3804992998 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0d54132b90 0x7f0d54125780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.874 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.872+0000 7f0d5bc4f640 1 --2- 172.21.15.70:0/3804992998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d5412df50 0x7f0d54125240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.874 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.872+0000 7f0d5bc4f640 1 --2- 172.21.15.70:0/3804992998 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0d5412d330 0x7f0d54124d00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.874 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.872+0000 7f0d5bc4f640 1 -- 172.21.15.70:0/3804992998 >> 172.21.15.70:0/3804992998 conn(0x7f0d540a0060 msgr2=0x7f0d540984d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:09.874 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.872+0000 7f0d5bc4f640 1 -- 172.21.15.70:0/3804992998 shutdown_connections 2024-09-13T15:44:09.874 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.872+0000 7f0d5bc4f640 1 -- 172.21.15.70:0/3804992998 wait complete. 2024-09-13T15:44:09.885 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T15:44:09.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T15:44:09.987 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.985+0000 7f8067080640 1 Processor -- start 2024-09-13T15:44:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.985+0000 7f8067080640 1 -- start start 2024-09-13T15:44:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f8067080640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f806012d4d0 0x7f806012d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f8067080640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f806012ddf0 0x7f8060129fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f8067080640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8060097620 con 0x7f806012a510 2024-09-13T15:44:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f8067080640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80600979f0 con 0x7f806012d4d0 2024-09-13T15:44:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f8067080640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806006d1f0 con 0x7f806012ddf0 2024-09-13T15:44:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f8064e1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f806012d4d0 0x7f806012d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f805ffff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f806012ddf0 0x7f8060129fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f806561d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f806012a510 0x7f8060134800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55496/0 (socket says 172.21.15.70:55496) 2024-09-13T15:44:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f8064e1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f806012d4d0 0x7f806012d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54238/0 (socket says 172.21.15.70:54238) 2024-09-13T15:44:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f806561d640 1 -- 172.21.15.70:0/2572803297 learned_addr learned my addr 172.21.15.70:0/2572803297 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f805ffff640 1 -- 172.21.15.70:0/2572803297 >> v2:172.21.15.70:3300/0 conn(0x7f806012d4d0 msgr2=0x7f806012d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f805ffff640 1 --2- 172.21.15.70:0/2572803297 >> v2:172.21.15.70:3300/0 conn(0x7f806012d4d0 0x7f806012d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f805ffff640 1 -- 172.21.15.70:0/2572803297 >> v1:172.21.15.70:6789/0 conn(0x7f806012a510 legacy=0x7f8060134800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f805f7fe640 1 -- 172.21.15.70:0/2572803297 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626996361 0 0) 0x7f8060097620 con 0x7f806012a510 2024-09-13T15:44:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f805ffff640 1 -- 172.21.15.70:0/2572803297 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806006db30 con 0x7f806012ddf0 2024-09-13T15:44:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.986+0000 7f8064e1c640 1 --2- 172.21.15.70:0/2572803297 >> v2:172.21.15.70:3300/0 conn(0x7f806012d4d0 0x7f806012d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:09.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.987+0000 7f805ffff640 1 --2- 172.21.15.70:0/2572803297 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f806012ddf0 0x7f8060129fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f804c0097a0 tx=0x7f804c02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca990bf2e164552 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:09.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.987+0000 7f805f7fe640 1 -- 172.21.15.70:0/2572803297 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c02adb0 con 0x7f806012ddf0 2024-09-13T15:44:09.992 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.987+0000 7f805f7fe640 1 -- 172.21.15.70:0/2572803297 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804c032440 con 0x7f806012ddf0 2024-09-13T15:44:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.987+0000 7f805f7fe640 1 -- 172.21.15.70:0/2572803297 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c0327d0 con 0x7f806012ddf0 2024-09-13T15:44:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.988+0000 7f8067080640 1 -- 172.21.15.70:0/2572803297 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f806012ddf0 msgr2=0x7f8060129fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.988+0000 7f8067080640 1 --2- 172.21.15.70:0/2572803297 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f806012ddf0 0x7f8060129fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f804c0097a0 tx=0x7f804c02a060 comp rx=0 tx=0).stop 2024-09-13T15:44:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.988+0000 7f8067080640 1 -- 172.21.15.70:0/2572803297 shutdown_connections 2024-09-13T15:44:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.988+0000 7f8067080640 1 --2- 172.21.15.70:0/2572803297 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f806012ddf0 0x7f8060129fd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.988+0000 7f8067080640 1 --2- 172.21.15.70:0/2572803297 >> v2:172.21.15.70:3300/0 conn(0x7f806012d4d0 0x7f806012d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.988+0000 7f8067080640 1 -- 172.21.15.70:0/2572803297 >> 172.21.15.70:0/2572803297 conn(0x7f806009fe20 msgr2=0x7f80600a0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.988+0000 7f8067080640 1 -- 172.21.15.70:0/2572803297 shutdown_connections 2024-09-13T15:44:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.988+0000 7f8067080640 1 -- 172.21.15.70:0/2572803297 wait complete. 2024-09-13T15:44:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.989+0000 7f8067080640 1 Processor -- start 2024-09-13T15:44:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.989+0000 7f8067080640 1 -- start start 2024-09-13T15:44:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.989+0000 7f8067080640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f806012a510 0x7f80601c2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.989+0000 7f8067080640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f806012d4d0 0x7f80601c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.989+0000 7f8067080640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f806012ddf0 0x7f806008bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.989+0000 7f8067080640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806008f210 con 0x7f806012d4d0 2024-09-13T15:44:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.989+0000 7f8067080640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8060127d30 con 0x7f806012ddf0 2024-09-13T15:44:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.989+0000 7f8067080640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f806008f5e0 con 0x7f806012a510 2024-09-13T15:44:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.989+0000 7f806561d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f806012ddf0 0x7f806008bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.989+0000 7f8064e1c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f806012a510 0x7f80601c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.989+0000 7f8064e1c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f806012a510 0x7f80601c2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:39790/0 (socket says 172.21.15.70:39790) 2024-09-13T15:44:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.989+0000 7f8064e1c640 1 -- 172.21.15.70:0/1327307124 learned_addr learned my addr 172.21.15.70:0/1327307124 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.989+0000 7f806561d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f806012ddf0 0x7f806008bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33376/0 (socket says 172.21.15.70:33376) 2024-09-13T15:44:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.989+0000 7f805ffff640 1 --2- 172.21.15.70:0/1327307124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f806012d4d0 0x7f80601c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.990+0000 7f806561d640 1 -- 172.21.15.70:0/1327307124 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f806012a510 msgr2=0x7f80601c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.990+0000 7f806561d640 1 --2- 172.21.15.70:0/1327307124 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f806012a510 0x7f80601c2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.990+0000 7f806561d640 1 -- 172.21.15.70:0/1327307124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f806012d4d0 msgr2=0x7f80601c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.990+0000 7f806561d640 1 --2- 172.21.15.70:0/1327307124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f806012d4d0 0x7f80601c2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:09.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.990+0000 7f806561d640 1 -- 172.21.15.70:0/1327307124 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8054009400 con 0x7f806012ddf0 2024-09-13T15:44:09.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.990+0000 7f805ffff640 1 --2- 172.21.15.70:0/1327307124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f806012d4d0 0x7f80601c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:09.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.990+0000 7f806561d640 1 --2- 172.21.15.70:0/1327307124 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f806012ddf0 0x7f806008bdc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f805800c780 tx=0x7f805800cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:09.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.990+0000 7f805d7fa640 1 -- 172.21.15.70:0/1327307124 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805800ce30 con 0x7f806012ddf0 2024-09-13T15:44:09.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.990+0000 7f805d7fa640 1 -- 172.21.15.70:0/1327307124 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805800a640 con 0x7f806012ddf0 2024-09-13T15:44:09.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.990+0000 7f8067080640 1 -- 172.21.15.70:0/1327307124 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804c009400 con 0x7f806012ddf0 2024-09-13T15:44:09.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.990+0000 7f805d7fa640 1 -- 172.21.15.70:0/1327307124 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805800a9f0 con 0x7f806012ddf0 2024-09-13T15:44:09.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.990+0000 7f8067080640 1 -- 172.21.15.70:0/1327307124 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80601c30b0 con 0x7f806012ddf0 2024-09-13T15:44:09.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.991+0000 7f8067080640 1 -- 172.21.15.70:0/1327307124 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f806009bd60 con 0x7f806012ddf0 2024-09-13T15:44:09.995 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.991+0000 7f805d7fa640 1 -- 172.21.15.70:0/1327307124 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f805800ab50 con 0x7f806012ddf0 2024-09-13T15:44:09.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.992+0000 7f805d7fa640 1 -- 172.21.15.70:0/1327307124 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f805805bdb0 con 0x7f806012ddf0 2024-09-13T15:44:09.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.992+0000 7f805d7fa640 1 -- 172.21.15.70:0/1327307124 --> v1:172.21.15.70:6804/853418053 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8044051aa0 con 0x7f804404ddf0 2024-09-13T15:44:09.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.992+0000 7f805d7fa640 1 -- 172.21.15.70:0/1327307124 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f805805e470 con 0x7f806012ddf0 2024-09-13T15:44:09.996 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:09.993+0000 7f805d7fa640 1 -- 172.21.15.70:0/1327307124 <== osd.1 v1:172.21.15.70:6804/853418053 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7f8044051aa0 con 0x7f804404ddf0 2024-09-13T15:44:10.005 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.004+0000 7f8067080640 1 -- 172.21.15.70:0/1327307124 --> v1:172.21.15.70:6804/853418053 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f80600a0320 con 0x7f804404ddf0 2024-09-13T15:44:10.006 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.005+0000 7f805d7fa640 1 -- 172.21.15.70:0/1327307124 <== osd.1 v1:172.21.15.70:6804/853418053 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (unknown 0 0 2175974996) 0x7f80600a0320 con 0x7f804404ddf0 2024-09-13T15:44:10.006 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-13T15:44:10.006 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-13T15:44:10.006 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-13T15:44:10.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.005+0000 7f8067080640 1 -- 172.21.15.70:0/1327307124 >> v1:172.21.15.70:6804/853418053 conn(0x7f804404ddf0 legacy=0x7f8044050230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.005+0000 7f8067080640 1 -- 172.21.15.70:0/1327307124 >> v1:172.21.15.70:6816/1229510400 conn(0x7f8044047560 legacy=0x7f8044049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.005+0000 7f8067080640 1 -- 172.21.15.70:0/1327307124 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f806012ddf0 msgr2=0x7f806008bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.005+0000 7f8067080640 1 --2- 172.21.15.70:0/1327307124 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f806012ddf0 0x7f806008bdc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f805800c780 tx=0x7f805800cc40 comp rx=0 tx=0).stop 2024-09-13T15:44:10.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.005+0000 7f806561d640 1 -- 172.21.15.70:0/1327307124 reap_dead start 2024-09-13T15:44:10.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.005+0000 7f8067080640 1 -- 172.21.15.70:0/1327307124 shutdown_connections 2024-09-13T15:44:10.007 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.005+0000 7f8067080640 1 -- 172.21.15.70:0/1327307124 >> 172.21.15.70:0/1327307124 conn(0x7f806009fe20 msgr2=0x7f806009a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:10.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.005+0000 7f8067080640 1 -- 172.21.15.70:0/1327307124 shutdown_connections 2024-09-13T15:44:10.008 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.005+0000 7f8067080640 1 -- 172.21.15.70:0/1327307124 wait complete. 2024-09-13T15:44:10.017 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T15:44:10.117 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.116+0000 7f63b4480640 1 Processor -- start 2024-09-13T15:44:10.118 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.116+0000 7f63b4480640 1 -- start start 2024-09-13T15:44:10.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b4480640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f63ac12d330 0x7f63ac12d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b4480640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f63ac12dd80 0x7f63ac129e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b4480640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ac06d5d0 con 0x7f63ac12a370 2024-09-13T15:44:10.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b4480640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0971b0 con 0x7f63ac12d330 2024-09-13T15:44:10.119 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b4480640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0979f0 con 0x7f63ac12dd80 2024-09-13T15:44:10.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b1a1b640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f63ac12dd80 0x7f63ac129e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b221c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f63ac12d330 0x7f63ac12d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b2a1d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f63ac12a370 0x7f63ac134660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55506/0 (socket says 172.21.15.70:55506) 2024-09-13T15:44:10.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b2a1d640 1 -- 172.21.15.70:0/643648186 learned_addr learned my addr 172.21.15.70:0/643648186 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:10.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b1a1b640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f63ac12dd80 0x7f63ac129e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:39802/0 (socket says 172.21.15.70:39802) 2024-09-13T15:44:10.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b1a1b640 1 -- 172.21.15.70:0/643648186 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f63ac12d330 msgr2=0x7f63ac12d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:10.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b1a1b640 1 --2- 172.21.15.70:0/643648186 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f63ac12d330 0x7f63ac12d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b121a640 1 -- 172.21.15.70:0/643648186 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422907082 0 0) 0x7f63ac06d5d0 con 0x7f63ac12a370 2024-09-13T15:44:10.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b1a1b640 1 -- 172.21.15.70:0/643648186 >> v1:172.21.15.70:6789/0 conn(0x7f63ac12a370 legacy=0x7f63ac134660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b1a1b640 1 -- 172.21.15.70:0/643648186 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac12e1a0 con 0x7f63ac12dd80 2024-09-13T15:44:10.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.117+0000 7f63b221c640 1 --2- 172.21.15.70:0/643648186 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f63ac12d330 0x7f63ac12d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:10.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.118+0000 7f63b1a1b640 1 --2- 172.21.15.70:0/643648186 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f63ac12dd80 0x7f63ac129e30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63a00097d0 tx=0x7f63a002a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=488bd1fe04e0fe6f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:10.120 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.118+0000 7f63b121a640 1 -- 172.21.15.70:0/643648186 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a002ad90 con 0x7f63ac12dd80 2024-09-13T15:44:10.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.118+0000 7f63b121a640 1 -- 172.21.15.70:0/643648186 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a0032440 con 0x7f63ac12dd80 2024-09-13T15:44:10.121 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.118+0000 7f63b121a640 1 -- 172.21.15.70:0/643648186 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a00327d0 con 0x7f63ac12dd80 2024-09-13T15:44:10.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.119+0000 7f63b4480640 1 -- 172.21.15.70:0/643648186 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f63ac12dd80 msgr2=0x7f63ac129e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.119+0000 7f63b4480640 1 --2- 172.21.15.70:0/643648186 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f63ac12dd80 0x7f63ac129e30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63a00097d0 tx=0x7f63a002a040 comp rx=0 tx=0).stop 2024-09-13T15:44:10.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.119+0000 7f63b4480640 1 -- 172.21.15.70:0/643648186 shutdown_connections 2024-09-13T15:44:10.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.119+0000 7f63b4480640 1 --2- 172.21.15.70:0/643648186 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f63ac12dd80 0x7f63ac129e30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.119+0000 7f63b4480640 1 --2- 172.21.15.70:0/643648186 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f63ac12d330 0x7f63ac12d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.119+0000 7f63b4480640 1 -- 172.21.15.70:0/643648186 >> 172.21.15.70:0/643648186 conn(0x7f63ac0a0060 msgr2=0x7f63ac0a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:10.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.119+0000 7f63b4480640 1 -- 172.21.15.70:0/643648186 shutdown_connections 2024-09-13T15:44:10.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.119+0000 7f63b4480640 1 -- 172.21.15.70:0/643648186 wait complete. 2024-09-13T15:44:10.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.120+0000 7f63b4480640 1 Processor -- start 2024-09-13T15:44:10.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.120+0000 7f63b4480640 1 -- start start 2024-09-13T15:44:10.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b4480640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63ac12a370 0x7f63ac1c2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b4480640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f63ac12d330 0x7f63ac1c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b4480640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f63ac12dd80 0x7f63ac08bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b4480640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac08f5a0 con 0x7f63ac12a370 2024-09-13T15:44:10.124 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b4480640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac08f1d0 con 0x7f63ac12d330 2024-09-13T15:44:10.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b4480640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac09a120 con 0x7f63ac12dd80 2024-09-13T15:44:10.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b221c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63ac12a370 0x7f63ac1c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b1a1b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f63ac12d330 0x7f63ac1c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b221c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63ac12a370 0x7f63ac1c2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54262/0 (socket says 172.21.15.70:54262) 2024-09-13T15:44:10.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b221c640 1 -- 172.21.15.70:0/2972770565 learned_addr learned my addr 172.21.15.70:0/2972770565 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:10.125 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b2a1d640 1 --2- 172.21.15.70:0/2972770565 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f63ac12dd80 0x7f63ac08bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b1a1b640 1 -- 172.21.15.70:0/2972770565 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f63ac12dd80 msgr2=0x7f63ac08bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b1a1b640 1 --2- 172.21.15.70:0/2972770565 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f63ac12dd80 0x7f63ac08bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b1a1b640 1 -- 172.21.15.70:0/2972770565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63ac12a370 msgr2=0x7f63ac1c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b1a1b640 1 --2- 172.21.15.70:0/2972770565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63ac12a370 0x7f63ac1c2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b1a1b640 1 -- 172.21.15.70:0/2972770565 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a8009400 con 0x7f63ac12d330 2024-09-13T15:44:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.121+0000 7f63b221c640 1 --2- 172.21.15.70:0/2972770565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63ac12a370 0x7f63ac1c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.122+0000 7f63b1a1b640 1 --2- 172.21.15.70:0/2972770565 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f63ac12d330 0x7f63ac1c29d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f63a0031000 tx=0x7f63a00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.122+0000 7f639effd640 1 -- 172.21.15.70:0/2972770565 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0004230 con 0x7f63ac12d330 2024-09-13T15:44:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.122+0000 7f639effd640 1 -- 172.21.15.70:0/2972770565 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a0004390 con 0x7f63ac12d330 2024-09-13T15:44:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.122+0000 7f63b4480640 1 -- 172.21.15.70:0/2972770565 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a0009400 con 0x7f63ac12d330 2024-09-13T15:44:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.122+0000 7f639effd640 1 -- 172.21.15.70:0/2972770565 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0003850 con 0x7f63ac12d330 2024-09-13T15:44:10.126 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.122+0000 7f63b4480640 1 -- 172.21.15.70:0/2972770565 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ac08e000 con 0x7f63ac12d330 2024-09-13T15:44:10.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.123+0000 7f63b4480640 1 -- 172.21.15.70:0/2972770565 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ac08e1f0 con 0x7f63ac12d330 2024-09-13T15:44:10.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.123+0000 7f639effd640 1 -- 172.21.15.70:0/2972770565 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f63a0039050 con 0x7f63ac12d330 2024-09-13T15:44:10.127 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.123+0000 7f639effd640 1 -- 172.21.15.70:0/2972770565 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f63a007cd60 con 0x7f63ac12d330 2024-09-13T15:44:10.128 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.127+0000 7f639effd640 1 -- 172.21.15.70:0/2972770565 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f63a0047cb0 con 0x7f63ac12d330 2024-09-13T15:44:10.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.265+0000 7f63b4480640 1 -- 172.21.15.70:0/2972770565 --> v1:172.21.15.70:6816/1229510400 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f63ac12d840 con 0x7f6390047560 2024-09-13T15:44:10.267 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.265+0000 7f639effd640 1 -- 172.21.15.70:0/2972770565 <== mgr.24114 v1:172.21.15.70:6816/1229510400 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (unknown 489605429 0 0) 0x7f63ac12d840 con 0x7f6390047560 2024-09-13T15:44:10.267 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T15:44:10.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.267+0000 7f63b4480640 1 -- 172.21.15.70:0/2972770565 >> v1:172.21.15.70:6816/1229510400 conn(0x7f6390047560 legacy=0x7f6390049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.267+0000 7f63b4480640 1 -- 172.21.15.70:0/2972770565 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f63ac12d330 msgr2=0x7f63ac1c29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.267+0000 7f63b4480640 1 --2- 172.21.15.70:0/2972770565 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f63ac12d330 0x7f63ac1c29d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f63a0031000 tx=0x7f63a00046c0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.267+0000 7f63b4480640 1 -- 172.21.15.70:0/2972770565 shutdown_connections 2024-09-13T15:44:10.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.267+0000 7f63b4480640 1 --2- 172.21.15.70:0/2972770565 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f63ac12dd80 0x7f63ac08bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.267+0000 7f63b4480640 1 --2- 172.21.15.70:0/2972770565 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f63ac12d330 0x7f63ac1c29d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.267+0000 7f63b4480640 1 --2- 172.21.15.70:0/2972770565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63ac12a370 0x7f63ac1c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.267+0000 7f63b4480640 1 -- 172.21.15.70:0/2972770565 >> 172.21.15.70:0/2972770565 conn(0x7f63ac0a0060 msgr2=0x7f63ac097cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:10.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.268+0000 7f63b4480640 1 -- 172.21.15.70:0/2972770565 shutdown_connections 2024-09-13T15:44:10.269 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.268+0000 7f63b4480640 1 -- 172.21.15.70:0/2972770565 wait complete. 2024-09-13T15:44:10.280 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T15:44:10.280 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T15:44:10.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.382+0000 7f35f183d640 1 Processor -- start 2024-09-13T15:44:10.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.382+0000 7f35f183d640 1 -- start start 2024-09-13T15:44:10.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.382+0000 7f35f183d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f35ec095300 0x7f35ec0956e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.382+0000 7f35f183d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35ec06c080 0x7f35ec06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.382+0000 7f35f183d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ec06e360 con 0x7f35ec095c20 2024-09-13T15:44:10.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.382+0000 7f35f183d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec06d300 con 0x7f35ec06c080 2024-09-13T15:44:10.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.382+0000 7f35f183d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec06da60 con 0x7f35ec095300 2024-09-13T15:44:10.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.382+0000 7f35ea7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f35ec095c20 0x7f35ec06b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55512/0 (socket says 172.21.15.70:55512) 2024-09-13T15:44:10.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.382+0000 7f35eb7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35ec06c080 0x7f35ec06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.382+0000 7f35ea7fc640 1 -- 172.21.15.70:0/821936387 learned_addr learned my addr 172.21.15.70:0/821936387 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:10.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.382+0000 7f35eaffd640 1 --2- 172.21.15.70:0/821936387 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f35ec095300 0x7f35ec0956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.383+0000 7f35e9ffb640 1 -- 172.21.15.70:0/821936387 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304663810 0 0) 0x7f35ec06e360 con 0x7f35ec095c20 2024-09-13T15:44:10.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.383+0000 7f35e9ffb640 1 -- 172.21.15.70:0/821936387 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d4003520 con 0x7f35ec095c20 2024-09-13T15:44:10.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.383+0000 7f35eb7fe640 1 -- 172.21.15.70:0/821936387 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f35ec095300 msgr2=0x7f35ec0956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:10.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.383+0000 7f35eb7fe640 1 --2- 172.21.15.70:0/821936387 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f35ec095300 0x7f35ec0956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.383+0000 7f35eb7fe640 1 -- 172.21.15.70:0/821936387 >> v1:172.21.15.70:6789/0 conn(0x7f35ec095c20 legacy=0x7f35ec06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.383+0000 7f35eb7fe640 1 -- 172.21.15.70:0/821936387 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ec06e080 con 0x7f35ec06c080 2024-09-13T15:44:10.385 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.383+0000 7f35eaffd640 1 --2- 172.21.15.70:0/821936387 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f35ec095300 0x7f35ec0956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:10.391 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.383+0000 7f35eb7fe640 1 --2- 172.21.15.70:0/821936387 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35ec06c080 0x7f35ec06c4b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009dc0 tx=0x7f35e402a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba403df4b649140f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.384+0000 7f35e9ffb640 1 -- 172.21.15.70:0/821936387 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e4007740 con 0x7f35ec06c080 2024-09-13T15:44:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.384+0000 7f35e9ffb640 1 -- 172.21.15.70:0/821936387 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e402d050 con 0x7f35ec06c080 2024-09-13T15:44:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.384+0000 7f35e9ffb640 1 -- 172.21.15.70:0/821936387 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e4007ac0 con 0x7f35ec06c080 2024-09-13T15:44:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.384+0000 7f35f183d640 1 -- 172.21.15.70:0/821936387 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35ec06c080 msgr2=0x7f35ec06c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.384+0000 7f35f183d640 1 --2- 172.21.15.70:0/821936387 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35ec06c080 0x7f35ec06c4b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009dc0 tx=0x7f35e402a510 comp rx=0 tx=0).stop 2024-09-13T15:44:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.384+0000 7f35f183d640 1 -- 172.21.15.70:0/821936387 shutdown_connections 2024-09-13T15:44:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.384+0000 7f35f183d640 1 --2- 172.21.15.70:0/821936387 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35ec06c080 0x7f35ec06c4b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.384+0000 7f35f183d640 1 --2- 172.21.15.70:0/821936387 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f35ec095300 0x7f35ec0956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.384+0000 7f35f183d640 1 -- 172.21.15.70:0/821936387 >> 172.21.15.70:0/821936387 conn(0x7f35ec098ab0 msgr2=0x7f35ec0963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.384+0000 7f35f183d640 1 -- 172.21.15.70:0/821936387 shutdown_connections 2024-09-13T15:44:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.384+0000 7f35f183d640 1 -- 172.21.15.70:0/821936387 wait complete. 2024-09-13T15:44:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.385+0000 7f35f183d640 1 Processor -- start 2024-09-13T15:44:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.385+0000 7f35f183d640 1 -- start start 2024-09-13T15:44:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35f183d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35ec06c080 0x7f35ec1bf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35f183d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35ec095300 0x7f35ec1bf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35f183d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f35ec095c20 0x7f35ec08ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35f183d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec090510 con 0x7f35ec095300 2024-09-13T15:44:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35f183d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec090140 con 0x7f35ec06c080 2024-09-13T15:44:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35f183d640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec08f110 con 0x7f35ec095c20 2024-09-13T15:44:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35eaffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35ec06c080 0x7f35ec1bf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35eb7fe640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f35ec095c20 0x7f35ec08ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35eaffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35ec06c080 0x7f35ec1bf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33426/0 (socket says 172.21.15.70:33426) 2024-09-13T15:44:10.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35eaffd640 1 -- 172.21.15.70:0/3650909747 learned_addr learned my addr 172.21.15.70:0/3650909747 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:10.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35ea7fc640 1 --2- 172.21.15.70:0/3650909747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35ec095300 0x7f35ec1bf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35eaffd640 1 -- 172.21.15.70:0/3650909747 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f35ec095c20 msgr2=0x7f35ec08ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35eaffd640 1 --2- 172.21.15.70:0/3650909747 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f35ec095c20 0x7f35ec08ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35eaffd640 1 -- 172.21.15.70:0/3650909747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35ec095300 msgr2=0x7f35ec1bf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35eaffd640 1 --2- 172.21.15.70:0/3650909747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35ec095300 0x7f35ec1bf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35eaffd640 1 -- 172.21.15.70:0/3650909747 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc009400 con 0x7f35ec06c080 2024-09-13T15:44:10.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35eb7fe640 1 --2- 172.21.15.70:0/3650909747 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f35ec095c20 0x7f35ec08ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:10.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35ea7fc640 1 --2- 172.21.15.70:0/3650909747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35ec095300 0x7f35ec1bf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:10.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35eaffd640 1 --2- 172.21.15.70:0/3650909747 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35ec06c080 0x7f35ec1bf210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00ebe0 tx=0x7f35dc00c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:10.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.386+0000 7f35d3fff640 1 -- 172.21.15.70:0/3650909747 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc009760 con 0x7f35ec06c080 2024-09-13T15:44:10.394 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.387+0000 7f35d3fff640 1 -- 172.21.15.70:0/3650909747 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc00cc30 con 0x7f35ec06c080 2024-09-13T15:44:10.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.387+0000 7f35d3fff640 1 -- 172.21.15.70:0/3650909747 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc00eee0 con 0x7f35ec06c080 2024-09-13T15:44:10.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.387+0000 7f35f183d640 1 -- 172.21.15.70:0/3650909747 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e4009a70 con 0x7f35ec06c080 2024-09-13T15:44:10.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.387+0000 7f35f183d640 1 -- 172.21.15.70:0/3650909747 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec1bfc90 con 0x7f35ec06c080 2024-09-13T15:44:10.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.387+0000 7f35f183d640 1 -- 172.21.15.70:0/3650909747 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35ec131f70 con 0x7f35ec06c080 2024-09-13T15:44:10.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.388+0000 7f35d3fff640 1 -- 172.21.15.70:0/3650909747 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f35dc013470 con 0x7f35ec06c080 2024-09-13T15:44:10.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.388+0000 7f35d3fff640 1 -- 172.21.15.70:0/3650909747 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f35dc05cbe0 con 0x7f35ec06c080 2024-09-13T15:44:10.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.388+0000 7f35d3fff640 1 -- 172.21.15.70:0/3650909747 --> v1:172.21.15.70:6808/2115999109 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35d8051aa0 con 0x7f35d804ddf0 2024-09-13T15:44:10.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.388+0000 7f35d3fff640 1 -- 172.21.15.70:0/3650909747 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35dc0612c0 con 0x7f35ec06c080 2024-09-13T15:44:10.395 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.389+0000 7f35d3fff640 1 -- 172.21.15.70:0/3650909747 <== osd.2 v1:172.21.15.70:6808/2115999109 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7f35d8051aa0 con 0x7f35d804ddf0 2024-09-13T15:44:10.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.401+0000 7f35f183d640 1 -- 172.21.15.70:0/3650909747 --> v1:172.21.15.70:6808/2115999109 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f35ec06c4b0 con 0x7f35d804ddf0 2024-09-13T15:44:10.402 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.401+0000 7f35d3fff640 1 -- 172.21.15.70:0/3650909747 <== osd.2 v1:172.21.15.70:6808/2115999109 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (unknown 0 0 2175974996) 0x7f35ec06c4b0 con 0x7f35d804ddf0 2024-09-13T15:44:10.402 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-13T15:44:10.403 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-13T15:44:10.403 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-13T15:44:10.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.401+0000 7f35f183d640 1 -- 172.21.15.70:0/3650909747 >> v1:172.21.15.70:6808/2115999109 conn(0x7f35d804ddf0 legacy=0x7f35d8050230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.401+0000 7f35f183d640 1 -- 172.21.15.70:0/3650909747 >> v1:172.21.15.70:6816/1229510400 conn(0x7f35d8047560 legacy=0x7f35d8049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.401+0000 7f35f183d640 1 -- 172.21.15.70:0/3650909747 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35ec06c080 msgr2=0x7f35ec1bf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.401+0000 7f35f183d640 1 --2- 172.21.15.70:0/3650909747 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f35ec06c080 0x7f35ec1bf210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00ebe0 tx=0x7f35dc00c520 comp rx=0 tx=0).stop 2024-09-13T15:44:10.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.401+0000 7f35eb7fe640 1 -- 172.21.15.70:0/3650909747 reap_dead start 2024-09-13T15:44:10.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.402+0000 7f35f183d640 1 -- 172.21.15.70:0/3650909747 shutdown_connections 2024-09-13T15:44:10.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.402+0000 7f35f183d640 1 -- 172.21.15.70:0/3650909747 >> 172.21.15.70:0/3650909747 conn(0x7f35ec098ab0 msgr2=0x7f35ec099840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:10.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.402+0000 7f35f183d640 1 -- 172.21.15.70:0/3650909747 shutdown_connections 2024-09-13T15:44:10.404 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.402+0000 7f35f183d640 1 -- 172.21.15.70:0/3650909747 wait complete. 2024-09-13T15:44:10.413 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T15:44:10.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e3559c640 1 Processor -- start 2024-09-13T15:44:10.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e3559c640 1 -- start start 2024-09-13T15:44:10.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e3559c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e3006c460 0x7f0e30132b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e3559c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e301330b0 0x7f0e301354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e3559c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3006d7d0 con 0x7f0e3006b970 2024-09-13T15:44:10.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e3559c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30097d40 con 0x7f0e3006c460 2024-09-13T15:44:10.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e3559c640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3006d120 con 0x7f0e301330b0 2024-09-13T15:44:10.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e2effd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0e3006b970 0x7f0e3006bd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55520/0 (socket says 172.21.15.70:55520) 2024-09-13T15:44:10.515 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e2effd640 1 -- 172.21.15.70:0/3434023844 learned_addr learned my addr 172.21.15.70:0/3434023844 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:10.516 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e2e7fc640 1 --2- 172.21.15.70:0/3434023844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e3006c460 0x7f0e30132b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.516 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e2f7fe640 1 --2- 172.21.15.70:0/3434023844 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e301330b0 0x7f0e301354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.516 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e2dffb640 1 -- 172.21.15.70:0/3434023844 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076623908 0 0) 0x7f0e3006d7d0 con 0x7f0e3006b970 2024-09-13T15:44:10.516 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e2dffb640 1 -- 172.21.15.70:0/3434023844 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e18003520 con 0x7f0e3006b970 2024-09-13T15:44:10.516 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e2e7fc640 1 -- 172.21.15.70:0/3434023844 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e301330b0 msgr2=0x7f0e301354a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:10.516 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e2e7fc640 1 --2- 172.21.15.70:0/3434023844 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e301330b0 0x7f0e301354a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.516 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e2e7fc640 1 -- 172.21.15.70:0/3434023844 >> v1:172.21.15.70:6789/0 conn(0x7f0e3006b970 legacy=0x7f0e3006bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.516 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e2e7fc640 1 -- 172.21.15.70:0/3434023844 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3006d980 con 0x7f0e3006c460 2024-09-13T15:44:10.516 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.513+0000 7f0e2f7fe640 1 --2- 172.21.15.70:0/3434023844 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e301330b0 0x7f0e301354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:10.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.514+0000 7f0e2e7fc640 1 --2- 172.21.15.70:0/3434023844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e3006c460 0x7f0e30132b70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e240097d0 tx=0x7f0e2402a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a19c5131b4be7fc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:10.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.514+0000 7f0e2dffb640 1 -- 172.21.15.70:0/3434023844 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2402ad90 con 0x7f0e3006c460 2024-09-13T15:44:10.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.514+0000 7f0e2dffb640 1 -- 172.21.15.70:0/3434023844 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e24032440 con 0x7f0e3006c460 2024-09-13T15:44:10.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.514+0000 7f0e2dffb640 1 -- 172.21.15.70:0/3434023844 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e240327d0 con 0x7f0e3006c460 2024-09-13T15:44:10.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.515+0000 7f0e3559c640 1 -- 172.21.15.70:0/3434023844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e3006c460 msgr2=0x7f0e30132b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.515+0000 7f0e3559c640 1 --2- 172.21.15.70:0/3434023844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e3006c460 0x7f0e30132b70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e240097d0 tx=0x7f0e2402a040 comp rx=0 tx=0).stop 2024-09-13T15:44:10.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.515+0000 7f0e3559c640 1 -- 172.21.15.70:0/3434023844 shutdown_connections 2024-09-13T15:44:10.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.515+0000 7f0e3559c640 1 --2- 172.21.15.70:0/3434023844 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e301330b0 0x7f0e301354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.515+0000 7f0e3559c640 1 --2- 172.21.15.70:0/3434023844 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e3006c460 0x7f0e30132b70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.520 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.515+0000 7f0e3559c640 1 -- 172.21.15.70:0/3434023844 >> 172.21.15.70:0/3434023844 conn(0x7f0e30098ae0 msgr2=0x7f0e300963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:10.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.515+0000 7f0e3559c640 1 -- 172.21.15.70:0/3434023844 shutdown_connections 2024-09-13T15:44:10.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.515+0000 7f0e3559c640 1 -- 172.21.15.70:0/3434023844 wait complete. 2024-09-13T15:44:10.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.516+0000 7f0e3559c640 1 Processor -- start 2024-09-13T15:44:10.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.516+0000 7f0e3559c640 1 -- start start 2024-09-13T15:44:10.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.516+0000 7f0e3559c640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e3006b970 0x7f0e30120a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.516+0000 7f0e3559c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e3006c460 0x7f0e30120fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.516+0000 7f0e3559c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e301330b0 0x7f0e301214f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.516+0000 7f0e3559c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30122890 con 0x7f0e3006c460 2024-09-13T15:44:10.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.516+0000 7f0e3559c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30123500 con 0x7f0e301330b0 2024-09-13T15:44:10.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.516+0000 7f0e3559c640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30122a90 con 0x7f0e3006b970 2024-09-13T15:44:10.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e2e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e3006c460 0x7f0e30120fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e2effd640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e3006b970 0x7f0e30120a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.521 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e2f7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e301330b0 0x7f0e301214f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e2e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e3006c460 0x7f0e30120fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54292/0 (socket says 172.21.15.70:54292) 2024-09-13T15:44:10.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e2e7fc640 1 -- 172.21.15.70:0/1630773276 learned_addr learned my addr 172.21.15.70:0/1630773276 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:10.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e2effd640 1 -- 172.21.15.70:0/1630773276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e301330b0 msgr2=0x7f0e301214f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e2effd640 1 --2- 172.21.15.70:0/1630773276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e301330b0 0x7f0e301214f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e2effd640 1 -- 172.21.15.70:0/1630773276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e3006c460 msgr2=0x7f0e30120fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e2effd640 1 --2- 172.21.15.70:0/1630773276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e3006c460 0x7f0e30120fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e2effd640 1 -- 172.21.15.70:0/1630773276 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e20009a70 con 0x7f0e3006b970 2024-09-13T15:44:10.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e2e7fc640 1 --2- 172.21.15.70:0/1630773276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e3006c460 0x7f0e30120fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:10.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e2effd640 1 --2- 172.21.15.70:0/1630773276 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e3006b970 0x7f0e30120a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800c430 tx=0x7f0e2800c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:10.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e177fe640 1 -- 172.21.15.70:0/1630773276 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28005ce0 con 0x7f0e3006b970 2024-09-13T15:44:10.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e177fe640 1 -- 172.21.15.70:0/1630773276 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e28005e40 con 0x7f0e3006b970 2024-09-13T15:44:10.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e3559c640 1 -- 172.21.15.70:0/1630773276 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e24009400 con 0x7f0e3006b970 2024-09-13T15:44:10.522 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e177fe640 1 -- 172.21.15.70:0/1630773276 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e280072d0 con 0x7f0e3006b970 2024-09-13T15:44:10.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.517+0000 7f0e3559c640 1 -- 172.21.15.70:0/1630773276 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e30121ff0 con 0x7f0e3006b970 2024-09-13T15:44:10.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.518+0000 7f0e177fe640 1 -- 172.21.15.70:0/1630773276 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f0e28007430 con 0x7f0e3006b970 2024-09-13T15:44:10.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.518+0000 7f0e3559c640 1 -- 172.21.15.70:0/1630773276 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3006bd50 con 0x7f0e3006b970 2024-09-13T15:44:10.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.519+0000 7f0e177fe640 1 -- 172.21.15.70:0/1630773276 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f0e2805be20 con 0x7f0e3006b970 2024-09-13T15:44:10.523 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.522+0000 7f0e177fe640 1 -- 172.21.15.70:0/1630773276 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f0e28039a90 con 0x7f0e3006b970 2024-09-13T15:44:10.663 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.662+0000 7f0e3559c640 1 -- 172.21.15.70:0/1630773276 --> v1:172.21.15.70:6816/1229510400 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e30134cc0 con 0x7f0e1c047560 2024-09-13T15:44:10.664 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.662+0000 7f0e177fe640 1 -- 172.21.15.70:0/1630773276 <== mgr.24114 v1:172.21.15.70:6816/1229510400 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (unknown 3049115190 0 0) 0x7f0e30134cc0 con 0x7f0e1c047560 2024-09-13T15:44:10.664 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T15:44:10.665 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.664+0000 7f0e3559c640 1 -- 172.21.15.70:0/1630773276 >> v1:172.21.15.70:6816/1229510400 conn(0x7f0e1c047560 legacy=0x7f0e1c049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.665 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.664+0000 7f0e3559c640 1 -- 172.21.15.70:0/1630773276 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e3006b970 msgr2=0x7f0e30120a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.664+0000 7f0e3559c640 1 --2- 172.21.15.70:0/1630773276 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e3006b970 0x7f0e30120a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800c430 tx=0x7f0e2800c830 comp rx=0 tx=0).stop 2024-09-13T15:44:10.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.664+0000 7f0e3559c640 1 -- 172.21.15.70:0/1630773276 shutdown_connections 2024-09-13T15:44:10.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.664+0000 7f0e3559c640 1 --2- 172.21.15.70:0/1630773276 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0e301330b0 0x7f0e301214f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.664+0000 7f0e3559c640 1 --2- 172.21.15.70:0/1630773276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e3006c460 0x7f0e30120fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.664+0000 7f0e3559c640 1 --2- 172.21.15.70:0/1630773276 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f0e3006b970 0x7f0e30120a70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.664+0000 7f0e3559c640 1 -- 172.21.15.70:0/1630773276 >> 172.21.15.70:0/1630773276 conn(0x7f0e30098ae0 msgr2=0x7f0e3009a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:10.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.664+0000 7f0e3559c640 1 -- 172.21.15.70:0/1630773276 shutdown_connections 2024-09-13T15:44:10.666 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.664+0000 7f0e3559c640 1 -- 172.21.15.70:0/1630773276 wait complete. 2024-09-13T15:44:10.677 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T15:44:10.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T15:44:10.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.777+0000 7f9cb89e9640 1 Processor -- start 2024-09-13T15:44:10.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.777+0000 7f9cb89e9640 1 -- start start 2024-09-13T15:44:10.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.778+0000 7f9cb89e9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cb412dfc0 0x7f9cb41327f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.778+0000 7f9cb89e9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9cb4132d30 0x7f9cb4135120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.778+0000 7f9cb89e9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cb4097620 con 0x7f9cb412d4d0 2024-09-13T15:44:10.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.778+0000 7f9cb89e9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb412e400 con 0x7f9cb4132d30 2024-09-13T15:44:10.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.778+0000 7f9cb89e9640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb40979f0 con 0x7f9cb412dfc0 2024-09-13T15:44:10.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.778+0000 7f9cb1d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cb412dfc0 0x7f9cb41327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.780 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.778+0000 7f9cb2d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9cb4132d30 0x7f9cb4135120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.778+0000 7f9cb259c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9cb412d4d0 0x7f9cb412d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55526/0 (socket says 172.21.15.70:55526) 2024-09-13T15:44:10.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.778+0000 7f9cb2d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9cb4132d30 0x7f9cb4135120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54294/0 (socket says 172.21.15.70:54294) 2024-09-13T15:44:10.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.778+0000 7f9cb259c640 1 -- 172.21.15.70:0/3277076011 learned_addr learned my addr 172.21.15.70:0/3277076011 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:10.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.778+0000 7f9cb2d9d640 1 -- 172.21.15.70:0/3277076011 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cb412dfc0 msgr2=0x7f9cb41327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:10.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.778+0000 7f9cb2d9d640 1 --2- 172.21.15.70:0/3277076011 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cb412dfc0 0x7f9cb41327f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.778+0000 7f9cb2d9d640 1 -- 172.21.15.70:0/3277076011 >> v1:172.21.15.70:6789/0 conn(0x7f9cb412d4d0 legacy=0x7f9cb412d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.779+0000 7f9cb2d9d640 1 -- 172.21.15.70:0/3277076011 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb406d1f0 con 0x7f9cb4132d30 2024-09-13T15:44:10.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.779+0000 7f9cb159a640 1 -- 172.21.15.70:0/3277076011 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79110820 0 0) 0x7f9cb4097620 con 0x7f9cb412d4d0 2024-09-13T15:44:10.781 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.779+0000 7f9cb1d9b640 1 --2- 172.21.15.70:0/3277076011 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cb412dfc0 0x7f9cb41327f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:10.784 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.779+0000 7f9cb2d9d640 1 --2- 172.21.15.70:0/3277076011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4132d30 0x7f9cb4135120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009d90 tx=0x7f9cac02a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585ab811f66bcc6e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.779+0000 7f9cb159a640 1 -- 172.21.15.70:0/3277076011 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac007740 con 0x7f9cb4132d30 2024-09-13T15:44:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.779+0000 7f9cb159a640 1 -- 172.21.15.70:0/3277076011 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cac02d050 con 0x7f9cb4132d30 2024-09-13T15:44:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.779+0000 7f9cb159a640 1 -- 172.21.15.70:0/3277076011 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac007ac0 con 0x7f9cb4132d30 2024-09-13T15:44:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.780+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3277076011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4132d30 msgr2=0x7f9cb4135120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.780+0000 7f9cb89e9640 1 --2- 172.21.15.70:0/3277076011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4132d30 0x7f9cb4135120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009d90 tx=0x7f9cac02a530 comp rx=0 tx=0).stop 2024-09-13T15:44:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.780+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3277076011 shutdown_connections 2024-09-13T15:44:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.780+0000 7f9cb89e9640 1 --2- 172.21.15.70:0/3277076011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4132d30 0x7f9cb4135120 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.780+0000 7f9cb89e9640 1 --2- 172.21.15.70:0/3277076011 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cb412dfc0 0x7f9cb41327f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.780+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3277076011 >> 172.21.15.70:0/3277076011 conn(0x7f9cb409fe20 msgr2=0x7f9cb40a0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.780+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3277076011 shutdown_connections 2024-09-13T15:44:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.780+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3277076011 wait complete. 2024-09-13T15:44:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.781+0000 7f9cb89e9640 1 Processor -- start 2024-09-13T15:44:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.781+0000 7f9cb89e9640 1 -- start start 2024-09-13T15:44:10.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.781+0000 7f9cb89e9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cb412d4d0 0x7f9cb41286b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.781+0000 7f9cb89e9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9cb412dfc0 0x7f9cb4128bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.781+0000 7f9cb89e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4132d30 0x7f9cb4129130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.781+0000 7f9cb89e9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb412a180 con 0x7f9cb4132d30 2024-09-13T15:44:10.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.781+0000 7f9cb89e9640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb412f090 con 0x7f9cb412d4d0 2024-09-13T15:44:10.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.781+0000 7f9cb89e9640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb412a8d0 con 0x7f9cb412dfc0 2024-09-13T15:44:10.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.781+0000 7f9cb259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cb412d4d0 0x7f9cb41286b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9cb259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cb412d4d0 0x7f9cb41286b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33446/0 (socket says 172.21.15.70:33446) 2024-09-13T15:44:10.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9cb259c640 1 -- 172.21.15.70:0/3994388299 learned_addr learned my addr 172.21.15.70:0/3994388299 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:10.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9cb1d9b640 1 --2- 172.21.15.70:0/3994388299 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9cb412dfc0 0x7f9cb4128bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9cb2d9d640 1 --2- 172.21.15.70:0/3994388299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4132d30 0x7f9cb4129130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9cb259c640 1 -- 172.21.15.70:0/3994388299 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9cb412dfc0 msgr2=0x7f9cb4128bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.786 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9cb259c640 1 --2- 172.21.15.70:0/3994388299 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f9cb412dfc0 0x7f9cb4128bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9cb259c640 1 -- 172.21.15.70:0/3994388299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4132d30 msgr2=0x7f9cb4129130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9cb259c640 1 --2- 172.21.15.70:0/3994388299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4132d30 0x7f9cb4129130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9cb259c640 1 -- 172.21.15.70:0/3994388299 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca8009400 con 0x7f9cb412d4d0 2024-09-13T15:44:10.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9cb2d9d640 1 --2- 172.21.15.70:0/3994388299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4132d30 0x7f9cb4129130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:10.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9cb259c640 1 --2- 172.21.15.70:0/3994388299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cb412d4d0 0x7f9cb41286b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400c4d0 tx=0x7f9ca400c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:10.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9ca2ffd640 1 -- 172.21.15.70:0/3994388299 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca40074e0 con 0x7f9cb412d4d0 2024-09-13T15:44:10.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9ca2ffd640 1 -- 172.21.15.70:0/3994388299 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca4007640 con 0x7f9cb412d4d0 2024-09-13T15:44:10.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3994388299 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac009a70 con 0x7f9cb412d4d0 2024-09-13T15:44:10.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9ca2ffd640 1 -- 172.21.15.70:0/3994388299 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4004a20 con 0x7f9cb412d4d0 2024-09-13T15:44:10.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.782+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3994388299 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb4132250 con 0x7f9cb412d4d0 2024-09-13T15:44:10.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.783+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3994388299 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9cb412d8b0 con 0x7f9cb412d4d0 2024-09-13T15:44:10.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.783+0000 7f9ca2ffd640 1 -- 172.21.15.70:0/3994388299 <== mon.1 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f9ca4004b80 con 0x7f9cb412d4d0 2024-09-13T15:44:10.787 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.784+0000 7f9ca2ffd640 1 -- 172.21.15.70:0/3994388299 <== mon.1 v2:172.21.15.70:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f9ca405be50 con 0x7f9cb412d4d0 2024-09-13T15:44:10.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.784+0000 7f9ca2ffd640 1 -- 172.21.15.70:0/3994388299 --> v1:172.21.15.70:6812/2095410362 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c94051aa0 con 0x7f9c9404ddf0 2024-09-13T15:44:10.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.784+0000 7f9ca2ffd640 1 -- 172.21.15.70:0/3994388299 <== mon.1 v2:172.21.15.70:3301/0 6 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ca405e510 con 0x7f9cb412d4d0 2024-09-13T15:44:10.788 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.784+0000 7f9ca2ffd640 1 -- 172.21.15.70:0/3994388299 <== osd.3 v1:172.21.15.70:6812/2095410362 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (unknown 0 0 1686565167) 0x7f9c94051aa0 con 0x7f9c9404ddf0 2024-09-13T15:44:10.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.796+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3994388299 --> v1:172.21.15.70:6812/2095410362 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9cb41358a0 con 0x7f9c9404ddf0 2024-09-13T15:44:10.798 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.797+0000 7f9ca2ffd640 1 -- 172.21.15.70:0/3994388299 <== osd.3 v1:172.21.15.70:6812/2095410362 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (unknown 0 0 2175974996) 0x7f9cb41358a0 con 0x7f9c9404ddf0 2024-09-13T15:44:10.798 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-13T15:44:10.798 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-13T15:44:10.798 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-13T15:44:10.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.797+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3994388299 >> v1:172.21.15.70:6812/2095410362 conn(0x7f9c9404ddf0 legacy=0x7f9c94050230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.797+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3994388299 >> v1:172.21.15.70:6816/1229510400 conn(0x7f9c94047560 legacy=0x7f9c94049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.797+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3994388299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cb412d4d0 msgr2=0x7f9cb41286b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.799 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.797+0000 7f9cb89e9640 1 --2- 172.21.15.70:0/3994388299 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9cb412d4d0 0x7f9cb41286b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ca400c4d0 tx=0x7f9ca400c990 comp rx=0 tx=0).stop 2024-09-13T15:44:10.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.797+0000 7f9cb2d9d640 1 -- 172.21.15.70:0/3994388299 reap_dead start 2024-09-13T15:44:10.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.797+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3994388299 shutdown_connections 2024-09-13T15:44:10.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.797+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3994388299 >> 172.21.15.70:0/3994388299 conn(0x7f9cb409fe20 msgr2=0x7f9cb40994f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:10.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.797+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3994388299 shutdown_connections 2024-09-13T15:44:10.800 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.797+0000 7f9cb89e9640 1 -- 172.21.15.70:0/3994388299 wait complete. 2024-09-13T15:44:10.808 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T15:44:10.906 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.905+0000 7fa117ae9640 1 Processor -- start 2024-09-13T15:44:10.907 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.905+0000 7fa117ae9640 1 -- start start 2024-09-13T15:44:10.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.906+0000 7fa117ae9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa11012df50 0x7fa110132650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.906+0000 7fa117ae9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa110132b90 0x7fa110134f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.906+0000 7fa117ae9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11006d5d0 con 0x7fa11012d330 2024-09-13T15:44:10.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.906+0000 7fa117ae9640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1100971b0 con 0x7fa110132b90 2024-09-13T15:44:10.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.906+0000 7fa117ae9640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa1100979f0 con 0x7fa11012df50 2024-09-13T15:44:10.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.906+0000 7fa115885640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa11012d330 0x7fa11012d840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55534/0 (socket says 172.21.15.70:55534) 2024-09-13T15:44:10.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.906+0000 7fa116086640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa110132b90 0x7fa110134f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.906+0000 7fa115885640 1 -- 172.21.15.70:0/3473327596 learned_addr learned my addr 172.21.15.70:0/3473327596 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:10.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.906+0000 7fa115084640 1 --2- 172.21.15.70:0/3473327596 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa11012df50 0x7fa110132650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.908 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.906+0000 7fa114883640 1 -- 172.21.15.70:0/3473327596 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365900690 0 0) 0x7fa11006d5d0 con 0x7fa11012d330 2024-09-13T15:44:10.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.906+0000 7fa116086640 1 -- 172.21.15.70:0/3473327596 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa11012df50 msgr2=0x7fa110132650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:10.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.906+0000 7fa116086640 1 --2- 172.21.15.70:0/3473327596 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa11012df50 0x7fa110132650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.906+0000 7fa116086640 1 -- 172.21.15.70:0/3473327596 >> v1:172.21.15.70:6789/0 conn(0x7fa11012d330 legacy=0x7fa11012d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.906+0000 7fa116086640 1 -- 172.21.15.70:0/3473327596 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa110097620 con 0x7fa110132b90 2024-09-13T15:44:10.909 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.906+0000 7fa115084640 1 --2- 172.21.15.70:0/3473327596 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa11012df50 0x7fa110132650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:10.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.907+0000 7fa116086640 1 --2- 172.21.15.70:0/3473327596 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa110132b90 0x7fa110134f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa108009dc0 tx=0x7fa10802a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df8925c8cbfe3ce2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:10.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.907+0000 7fa114883640 1 -- 172.21.15.70:0/3473327596 <== mon.1 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa108007740 con 0x7fa110132b90 2024-09-13T15:44:10.912 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.907+0000 7fa114883640 1 -- 172.21.15.70:0/3473327596 <== mon.1 v2:172.21.15.70:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10802d050 con 0x7fa110132b90 2024-09-13T15:44:10.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.907+0000 7fa114883640 1 -- 172.21.15.70:0/3473327596 <== mon.1 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa108007ac0 con 0x7fa110132b90 2024-09-13T15:44:10.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.908+0000 7fa117ae9640 1 -- 172.21.15.70:0/3473327596 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa110132b90 msgr2=0x7fa110134f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.908+0000 7fa117ae9640 1 --2- 172.21.15.70:0/3473327596 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa110132b90 0x7fa110134f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa108009dc0 tx=0x7fa10802a510 comp rx=0 tx=0).stop 2024-09-13T15:44:10.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.908+0000 7fa117ae9640 1 -- 172.21.15.70:0/3473327596 shutdown_connections 2024-09-13T15:44:10.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.908+0000 7fa117ae9640 1 --2- 172.21.15.70:0/3473327596 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa110132b90 0x7fa110134f80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.908+0000 7fa117ae9640 1 --2- 172.21.15.70:0/3473327596 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa11012df50 0x7fa110132650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.908+0000 7fa117ae9640 1 -- 172.21.15.70:0/3473327596 >> 172.21.15.70:0/3473327596 conn(0x7fa1100a0060 msgr2=0x7fa1100a0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:10.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.908+0000 7fa117ae9640 1 -- 172.21.15.70:0/3473327596 shutdown_connections 2024-09-13T15:44:10.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.908+0000 7fa117ae9640 1 -- 172.21.15.70:0/3473327596 wait complete. 2024-09-13T15:44:10.913 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.909+0000 7fa117ae9640 1 Processor -- start 2024-09-13T15:44:10.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.909+0000 7fa117ae9640 1 -- start start 2024-09-13T15:44:10.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.909+0000 7fa117ae9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11012d330 0x7fa110124d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.909+0000 7fa117ae9640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa11012df50 0x7fa110125240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.909+0000 7fa117ae9640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa110132b90 0x7fa110125780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:10.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.909+0000 7fa117ae9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa110127060 con 0x7fa11012d330 2024-09-13T15:44:10.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.909+0000 7fa117ae9640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa110127840 con 0x7fa11012df50 2024-09-13T15:44:10.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.909+0000 7fa117ae9640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa110127c10 con 0x7fa110132b90 2024-09-13T15:44:10.914 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.909+0000 7fa115885640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11012d330 0x7fa110124d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.909+0000 7fa116086640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa110132b90 0x7fa110125780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.909+0000 7fa115885640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11012d330 0x7fa110124d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54318/0 (socket says 172.21.15.70:54318) 2024-09-13T15:44:10.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.909+0000 7fa115885640 1 -- 172.21.15.70:0/423518865 learned_addr learned my addr 172.21.15.70:0/423518865 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:10.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.910+0000 7fa115084640 1 --2- 172.21.15.70:0/423518865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa11012df50 0x7fa110125240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:10.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.910+0000 7fa115885640 1 -- 172.21.15.70:0/423518865 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa110132b90 msgr2=0x7fa110125780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.910+0000 7fa115885640 1 --2- 172.21.15.70:0/423518865 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa110132b90 0x7fa110125780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.910+0000 7fa115885640 1 -- 172.21.15.70:0/423518865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa11012df50 msgr2=0x7fa110125240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:10.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.910+0000 7fa115885640 1 --2- 172.21.15.70:0/423518865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa11012df50 0x7fa110125240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:10.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.910+0000 7fa115885640 1 -- 172.21.15.70:0/423518865 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa100009400 con 0x7fa11012d330 2024-09-13T15:44:10.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.910+0000 7fa116086640 1 --2- 172.21.15.70:0/423518865 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa110132b90 0x7fa110125780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:10.915 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.910+0000 7fa115885640 1 --2- 172.21.15.70:0/423518865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11012d330 0x7fa110124d00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00c4a0 tx=0x7fa10c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:10.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.910+0000 7fa1067fc640 1 -- 172.21.15.70:0/423518865 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0074e0 con 0x7fa11012d330 2024-09-13T15:44:10.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.910+0000 7fa1067fc640 1 -- 172.21.15.70:0/423518865 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c007640 con 0x7fa11012d330 2024-09-13T15:44:10.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.910+0000 7fa117ae9640 1 -- 172.21.15.70:0/423518865 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa108009a70 con 0x7fa11012d330 2024-09-13T15:44:10.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.910+0000 7fa1067fc640 1 -- 172.21.15.70:0/423518865 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c004a00 con 0x7fa11012d330 2024-09-13T15:44:10.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.910+0000 7fa117ae9640 1 -- 172.21.15.70:0/423518865 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa110126870 con 0x7fa11012d330 2024-09-13T15:44:10.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.911+0000 7fa1067fc640 1 -- 172.21.15.70:0/423518865 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7fa10c004b60 con 0x7fa11012d330 2024-09-13T15:44:10.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.911+0000 7fa117ae9640 1 -- 172.21.15.70:0/423518865 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11012d840 con 0x7fa11012d330 2024-09-13T15:44:10.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.912+0000 7fa1067fc640 1 -- 172.21.15.70:0/423518865 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7fa10c05bb70 con 0x7fa11012d330 2024-09-13T15:44:10.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:10.915+0000 7fa1067fc640 1 -- 172.21.15.70:0/423518865 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7fa10c0397e0 con 0x7fa11012d330 2024-09-13T15:44:11.075 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.074+0000 7fa117ae9640 1 -- 172.21.15.70:0/423518865 --> v1:172.21.15.70:6816/1229510400 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa11009c450 con 0x7fa0f4047560 2024-09-13T15:44:11.076 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.074+0000 7fa1067fc640 1 -- 172.21.15.70:0/423518865 <== mgr.24114 v1:172.21.15.70:6816/1229510400 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (unknown 3526570807 0 0) 0x7fa11009c450 con 0x7fa0f4047560 2024-09-13T15:44:11.076 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T15:44:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.076+0000 7fa117ae9640 1 -- 172.21.15.70:0/423518865 >> v1:172.21.15.70:6816/1229510400 conn(0x7fa0f4047560 legacy=0x7fa0f4049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.076+0000 7fa117ae9640 1 -- 172.21.15.70:0/423518865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11012d330 msgr2=0x7fa110124d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.076+0000 7fa117ae9640 1 --2- 172.21.15.70:0/423518865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11012d330 0x7fa110124d00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00c4a0 tx=0x7fa10c00c960 comp rx=0 tx=0).stop 2024-09-13T15:44:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.076+0000 7fa117ae9640 1 -- 172.21.15.70:0/423518865 shutdown_connections 2024-09-13T15:44:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.076+0000 7fa117ae9640 1 --2- 172.21.15.70:0/423518865 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7fa110132b90 0x7fa110125780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.076+0000 7fa117ae9640 1 --2- 172.21.15.70:0/423518865 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa11012df50 0x7fa110125240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.076+0000 7fa117ae9640 1 --2- 172.21.15.70:0/423518865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11012d330 0x7fa110124d00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:11.079 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.076+0000 7fa117ae9640 1 -- 172.21.15.70:0/423518865 >> 172.21.15.70:0/423518865 conn(0x7fa1100a0060 msgr2=0x7fa110098a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:11.079 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.076+0000 7fa117ae9640 1 -- 172.21.15.70:0/423518865 shutdown_connections 2024-09-13T15:44:11.079 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.076+0000 7fa117ae9640 1 -- 172.21.15.70:0/423518865 wait complete. 2024-09-13T15:44:11.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T15:44:11.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.184+0000 7feb18d18640 1 Processor -- start 2024-09-13T15:44:11.185 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.184+0000 7feb18d18640 1 -- start start 2024-09-13T15:44:11.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.184+0000 7feb18d18640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feb1406ddf0 0x7feb1412f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:11.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.184+0000 7feb18d18640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feb1412f960 0x7feb14131d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:11.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.184+0000 7feb18d18640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb1406e3a0 con 0x7feb140953e0 2024-09-13T15:44:11.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.184+0000 7feb18d18640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb14095ed0 con 0x7feb1406ddf0 2024-09-13T15:44:11.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.184+0000 7feb18d18640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb14097d40 con 0x7feb1412f960 2024-09-13T15:44:11.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.184+0000 7feb11d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feb1406ddf0 0x7feb1412f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:11.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.184+0000 7feb1259c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7feb140953e0 0x7feb140957c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55546/0 (socket says 172.21.15.70:55546) 2024-09-13T15:44:11.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.184+0000 7feb12d9d640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feb1412f960 0x7feb14131d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:11.186 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.184+0000 7feb1259c640 1 -- 172.21.15.70:0/3878848030 learned_addr learned my addr 172.21.15.70:0/3878848030 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:11.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.185+0000 7feb11d9b640 1 -- 172.21.15.70:0/3878848030 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feb1412f960 msgr2=0x7feb14131d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:11.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.185+0000 7feb11d9b640 1 --2- 172.21.15.70:0/3878848030 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feb1412f960 0x7feb14131d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:11.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.185+0000 7feb11d9b640 1 -- 172.21.15.70:0/3878848030 >> v1:172.21.15.70:6789/0 conn(0x7feb140953e0 legacy=0x7feb140957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:11.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.185+0000 7feb11d9b640 1 -- 172.21.15.70:0/3878848030 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1406f110 con 0x7feb1406ddf0 2024-09-13T15:44:11.187 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.185+0000 7feb12d9d640 1 --2- 172.21.15.70:0/3878848030 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feb1412f960 0x7feb14131d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:11.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.185+0000 7feb11d9b640 1 --2- 172.21.15.70:0/3878848030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb1406ddf0 0x7feb1412f420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feb040097a0 tx=0x7feb0402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0665ab93e2e5c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.185+0000 7feb1159a640 1 -- 172.21.15.70:0/3878848030 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0402adb0 con 0x7feb1406ddf0 2024-09-13T15:44:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.185+0000 7feb1159a640 1 -- 172.21.15.70:0/3878848030 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb04032440 con 0x7feb1406ddf0 2024-09-13T15:44:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.185+0000 7feb1159a640 1 -- 172.21.15.70:0/3878848030 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb040327d0 con 0x7feb1406ddf0 2024-09-13T15:44:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.186+0000 7feb18d18640 1 -- 172.21.15.70:0/3878848030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb1406ddf0 msgr2=0x7feb1412f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.186+0000 7feb18d18640 1 --2- 172.21.15.70:0/3878848030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb1406ddf0 0x7feb1412f420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feb040097a0 tx=0x7feb0402a060 comp rx=0 tx=0).stop 2024-09-13T15:44:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.186+0000 7feb18d18640 1 -- 172.21.15.70:0/3878848030 shutdown_connections 2024-09-13T15:44:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.186+0000 7feb18d18640 1 --2- 172.21.15.70:0/3878848030 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feb1412f960 0x7feb14131d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.186+0000 7feb18d18640 1 --2- 172.21.15.70:0/3878848030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb1406ddf0 0x7feb1412f420 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.186+0000 7feb18d18640 1 -- 172.21.15.70:0/3878848030 >> 172.21.15.70:0/3878848030 conn(0x7feb14098b10 msgr2=0x7feb140963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.186+0000 7feb18d18640 1 -- 172.21.15.70:0/3878848030 shutdown_connections 2024-09-13T15:44:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.186+0000 7feb18d18640 1 -- 172.21.15.70:0/3878848030 wait complete. 2024-09-13T15:44:11.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.187+0000 7feb18d18640 1 Processor -- start 2024-09-13T15:44:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.187+0000 7feb18d18640 1 -- start start 2024-09-13T15:44:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.187+0000 7feb18d18640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feb1406ddf0 0x7feb14124600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.187+0000 7feb18d18640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feb140953e0 0x7feb14094450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.187+0000 7feb18d18640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb1412f960 0x7feb14094990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.187+0000 7feb18d18640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb14131860 con 0x7feb1412f960 2024-09-13T15:44:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.187+0000 7feb18d18640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb14124b40 con 0x7feb1406ddf0 2024-09-13T15:44:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.187+0000 7feb18d18640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb14121f90 con 0x7feb140953e0 2024-09-13T15:44:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb11d9b640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feb140953e0 0x7feb14094450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feb1406ddf0 0x7feb14124600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb12d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb1412f960 0x7feb14094990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb11d9b640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feb140953e0 0x7feb14094450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:39888/0 (socket says 172.21.15.70:39888) 2024-09-13T15:44:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb1259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feb1406ddf0 0x7feb14124600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:33474/0 (socket says 172.21.15.70:33474) 2024-09-13T15:44:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb11d9b640 1 -- 172.21.15.70:0/2929242343 learned_addr learned my addr 172.21.15.70:0/2929242343 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb12d9d640 1 -- 172.21.15.70:0/2929242343 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feb140953e0 msgr2=0x7feb14094450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb12d9d640 1 --2- 172.21.15.70:0/2929242343 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feb140953e0 0x7feb14094450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb12d9d640 1 -- 172.21.15.70:0/2929242343 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feb1406ddf0 msgr2=0x7feb14124600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb12d9d640 1 --2- 172.21.15.70:0/2929242343 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feb1406ddf0 0x7feb14124600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb12d9d640 1 -- 172.21.15.70:0/2929242343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c009a70 con 0x7feb1412f960 2024-09-13T15:44:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb12d9d640 1 --2- 172.21.15.70:0/2929242343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb1412f960 0x7feb14094990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7feb1406eae0 tx=0x7feb0c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb0affd640 1 -- 172.21.15.70:0/2929242343 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c014480 con 0x7feb1412f960 2024-09-13T15:44:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb0affd640 1 -- 172.21.15.70:0/2929242343 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0c00f040 con 0x7feb1412f960 2024-09-13T15:44:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb18d18640 1 -- 172.21.15.70:0/2929242343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb04009400 con 0x7feb1412f960 2024-09-13T15:44:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb0affd640 1 -- 172.21.15.70:0/2929242343 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c014840 con 0x7feb1412f960 2024-09-13T15:44:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.188+0000 7feb18d18640 1 -- 172.21.15.70:0/2929242343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb14122d70 con 0x7feb1412f960 2024-09-13T15:44:11.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.189+0000 7feb0affd640 1 -- 172.21.15.70:0/2929242343 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7feb0c0149a0 con 0x7feb1412f960 2024-09-13T15:44:11.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.189+0000 7feb18d18640 1 -- 172.21.15.70:0/2929242343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb140957c0 con 0x7feb1412f960 2024-09-13T15:44:11.193 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.190+0000 7feb0affd640 1 -- 172.21.15.70:0/2929242343 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7feb0c05d7f0 con 0x7feb1412f960 2024-09-13T15:44:11.194 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.193+0000 7feb0affd640 1 -- 172.21.15.70:0/2929242343 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7feb0c03b460 con 0x7feb1412f960 2024-09-13T15:44:11.327 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.325+0000 7feb18d18640 1 -- 172.21.15.70:0/2929242343 --> v1:172.21.15.70:6816/1229510400 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb14066fa0 con 0x7feaf4047560 2024-09-13T15:44:11.328 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.327+0000 7feb0affd640 1 -- 172.21.15.70:0/2929242343 <== mgr.24114 v1:172.21.15.70:6816/1229510400 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34765 (unknown 2965378022 0 2988304794) 0x7feb14066fa0 con 0x7feaf4047560 2024-09-13T15:44:11.328 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:44:11.329 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-13T15:44:11.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.328+0000 7feb18d18640 1 -- 172.21.15.70:0/2929242343 >> v1:172.21.15.70:6816/1229510400 conn(0x7feaf4047560 legacy=0x7feaf4049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:11.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.328+0000 7feb18d18640 1 -- 172.21.15.70:0/2929242343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb1412f960 msgr2=0x7feb14094990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:11.330 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.328+0000 7feb18d18640 1 --2- 172.21.15.70:0/2929242343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb1412f960 0x7feb14094990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7feb1406eae0 tx=0x7feb0c00ef90 comp rx=0 tx=0).stop 2024-09-13T15:44:11.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.328+0000 7feb18d18640 1 -- 172.21.15.70:0/2929242343 shutdown_connections 2024-09-13T15:44:11.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.328+0000 7feb18d18640 1 --2- 172.21.15.70:0/2929242343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb1412f960 0x7feb14094990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:11.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.328+0000 7feb18d18640 1 --2- 172.21.15.70:0/2929242343 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7feb140953e0 0x7feb14094450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:11.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.328+0000 7feb18d18640 1 --2- 172.21.15.70:0/2929242343 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feb1406ddf0 0x7feb14124600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:11.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.328+0000 7feb18d18640 1 -- 172.21.15.70:0/2929242343 >> 172.21.15.70:0/2929242343 conn(0x7feb14098b10 msgr2=0x7feb14124cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:11.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.328+0000 7feb18d18640 1 -- 172.21.15.70:0/2929242343 shutdown_connections 2024-09-13T15:44:11.331 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:11.328+0000 7feb18d18640 1 -- 172.21.15.70:0/2929242343 wait complete. 2024-09-13T15:44:11.340 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-13T15:44:09.541264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":129004,"kb_used_data":2224,"kb_used_omap":17,"kb_used_meta":126702,"kb_avail":377358356,"statfs":{"total":386547056640,"available":386414956544,"internally_reserved":0,"allocated":2277376,"data_stored":1562038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":129743093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001481"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599825+0000","last_change":"2024-09-13T15:43:56.791803+0000","last_active":"2024-09-13T15:44:01.599825+0000","last_peered":"2024-09-13T15:44:01.599825+0000","last_clean":"2024-09-13T15:44:01.599825+0000","last_became_active":"2024-09-13T15:43:56.791454+0000","last_became_peered":"2024-09-13T15:43:56.791454+0000","last_unstale":"2024-09-13T15:44:01.599825+0000","last_undegraded":"2024-09-13T15:44:01.599825+0000","last_fullsized":"2024-09-13T15:44:01.599825+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:38.078282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035985100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599965+0000","last_change":"2024-09-13T15:43:54.574295+0000","last_active":"2024-09-13T15:44:01.599965+0000","last_peered":"2024-09-13T15:44:01.599965+0000","last_clean":"2024-09-13T15:44:01.599965+0000","last_became_active":"2024-09-13T15:43:54.573978+0000","last_became_peered":"2024-09-13T15:43:54.573978+0000","last_unstale":"2024-09-13T15:44:01.599965+0000","last_undegraded":"2024-09-13T15:44:01.599965+0000","last_fullsized":"2024-09-13T15:44:01.599965+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:09.974326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054553999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599736+0000","last_change":"2024-09-13T15:43:56.792140+0000","last_active":"2024-09-13T15:44:01.599736+0000","last_peered":"2024-09-13T15:44:01.599736+0000","last_clean":"2024-09-13T15:44:01.599736+0000","last_became_active":"2024-09-13T15:43:56.791789+0000","last_became_peered":"2024-09-13T15:43:56.791789+0000","last_unstale":"2024-09-13T15:44:01.599736+0000","last_undegraded":"2024-09-13T15:44:01.599736+0000","last_fullsized":"2024-09-13T15:44:01.599736+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:51.347592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011766770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599970+0000","last_change":"2024-09-13T15:43:54.573858+0000","last_active":"2024-09-13T15:44:01.599970+0000","last_peered":"2024-09-13T15:44:01.599970+0000","last_clean":"2024-09-13T15:44:01.599970+0000","last_became_active":"2024-09-13T15:43:54.573540+0000","last_became_peered":"2024-09-13T15:43:54.573540+0000","last_unstale":"2024-09-13T15:44:01.599970+0000","last_undegraded":"2024-09-13T15:44:01.599970+0000","last_fullsized":"2024-09-13T15:44:01.599970+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:50.925568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024452499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599730+0000","last_change":"2024-09-13T15:43:56.584865+0000","last_active":"2024-09-13T15:44:01.599730+0000","last_peered":"2024-09-13T15:44:01.599730+0000","last_clean":"2024-09-13T15:44:01.599730+0000","last_became_active":"2024-09-13T15:43:56.584517+0000","last_became_peered":"2024-09-13T15:43:56.584517+0000","last_unstale":"2024-09-13T15:44:01.599730+0000","last_undegraded":"2024-09-13T15:44:01.599730+0000","last_fullsized":"2024-09-13T15:44:01.599730+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:29:26.245037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094542300000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.991513+0000","last_change":"2024-09-13T15:43:56.582866+0000","last_active":"2024-09-13T15:44:01.991513+0000","last_peered":"2024-09-13T15:44:01.991513+0000","last_clean":"2024-09-13T15:44:01.991513+0000","last_became_active":"2024-09-13T15:43:56.582550+0000","last_became_peered":"2024-09-13T15:43:56.582550+0000","last_unstale":"2024-09-13T15:44:01.991513+0000","last_undegraded":"2024-09-13T15:44:01.991513+0000","last_fullsized":"2024-09-13T15:44:01.991513+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:26.074766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089033199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599643+0000","last_change":"2024-09-13T15:43:56.585264+0000","last_active":"2024-09-13T15:44:01.599643+0000","last_peered":"2024-09-13T15:44:01.599643+0000","last_clean":"2024-09-13T15:44:01.599643+0000","last_became_active":"2024-09-13T15:43:56.584633+0000","last_became_peered":"2024-09-13T15:43:56.584633+0000","last_unstale":"2024-09-13T15:44:01.599643+0000","last_undegraded":"2024-09-13T15:44:01.599643+0000","last_fullsized":"2024-09-13T15:44:01.599643+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:29.252314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034895899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599737+0000","last_change":"2024-09-13T15:43:56.583583+0000","last_active":"2024-09-13T15:44:01.599737+0000","last_peered":"2024-09-13T15:44:01.599737+0000","last_clean":"2024-09-13T15:44:01.599737+0000","last_became_active":"2024-09-13T15:43:56.582549+0000","last_became_peered":"2024-09-13T15:43:56.582549+0000","last_unstale":"2024-09-13T15:44:01.599737+0000","last_undegraded":"2024-09-13T15:44:01.599737+0000","last_fullsized":"2024-09-13T15:44:01.599737+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:26.707719+0000","last_clean_scrub_stamp":"2024-09-13T15:42:26.707719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:58:03.411133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010276560000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'63","reported_seq":140,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T15:44:01.599581+0000","last_change":"2024-09-13T15:43:54.574397+0000","last_active":"2024-09-13T15:44:01.599581+0000","last_peered":"2024-09-13T15:44:01.599581+0000","last_clean":"2024-09-13T15:44:01.599581+0000","last_became_active":"2024-09-13T15:43:54.574044+0000","last_became_peered":"2024-09-13T15:43:54.574044+0000","last_unstale":"2024-09-13T15:44:01.599581+0000","last_undegraded":"2024-09-13T15:44:01.599581+0000","last_fullsized":"2024-09-13T15:44:01.599581+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:42:21.689345+0000","last_clean_scrub_stamp":"2024-09-13T15:42:21.689345+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:48:10.619565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":28,"seq":120259084292,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":268,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":274432,"data_stored":95335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.215}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]}]},{"osd":2,"up_from":27,"seq":115964116997,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":264,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":270336,"data_stored":95316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]}]},{"osd":1,"up_from":26,"seq":111669149701,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32544,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339296,"statfs":{"total":96636764160,"available":96603439104,"internally_reserved":0,"allocated":868352,"data_stored":685703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":32435774},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]}]},{"osd":0,"up_from":25,"seq":107374182405,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":844,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":864256,"data_stored":685684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.245}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T15:44:11.343 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-13T15:42:26.707719+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=42, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=44, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T15:44:11.344 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-13T15:42:26.707719+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=42, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=44, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T15:44:11.344 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T15:42:26.707719+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=42, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=44, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T15:44:11.344 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-13T15:42:26.707719+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=42, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=44, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T15:44:11.344 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-13T15:42:26.707719+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=42, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=44, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T15:44:11.344 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-13T15:42:26.707719+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=42, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=44, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T15:44:11.344 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T15:42:26.707719+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=42, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=44, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T15:44:11.344 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-13T15:42:26.707719+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=42, tm_sec=26, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=44, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T15:44:11.344 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T15:42:21.689345+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=42, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=15, tm_min=44, tm_sec=8, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T15:44:11.344 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T15:44:31.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T15:44:31.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.453+0000 7f34ff965640 1 Processor -- start 2024-09-13T15:44:31.455 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.453+0000 7f34ff965640 1 -- start start 2024-09-13T15:44:31.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34ff965640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34f80953e0 0x7f34f80957c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:31.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34ff965640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f34f806d190 0x7f34f8132440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:31.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34ff965640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f806e3a0 con 0x7f34f806ddf0 2024-09-13T15:44:31.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34ff965640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f8097d40 con 0x7f34f80953e0 2024-09-13T15:44:31.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34ff965640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34f806f110 con 0x7f34f806d190 2024-09-13T15:44:31.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34fd701640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34f80953e0 0x7f34f80957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:31.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34fcf00640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f34f806ddf0 0x7f34f806ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37802/0 (socket says 172.21.15.70:37802) 2024-09-13T15:44:31.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34fdf02640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f34f806d190 0x7f34f8132440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:31.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34fcf00640 1 -- 172.21.15.70:0/2286306876 learned_addr learned my addr 172.21.15.70:0/2286306876 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-09-13T15:44:31.458 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34effff640 1 -- 172.21.15.70:0/2286306876 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431828975 0 0) 0x7f34f806e3a0 con 0x7f34f806ddf0 2024-09-13T15:44:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34effff640 1 -- 172.21.15.70:0/2286306876 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e4003520 con 0x7f34f806ddf0 2024-09-13T15:44:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34fdf02640 1 -- 172.21.15.70:0/2286306876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34f80953e0 msgr2=0x7f34f80957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34fdf02640 1 --2- 172.21.15.70:0/2286306876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34f80953e0 0x7f34f80957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34fdf02640 1 -- 172.21.15.70:0/2286306876 >> v1:172.21.15.70:6789/0 conn(0x7f34f806ddf0 legacy=0x7f34f806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34fdf02640 1 -- 172.21.15.70:0/2286306876 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f806eb70 con 0x7f34f806d190 2024-09-13T15:44:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.454+0000 7f34fd701640 1 --2- 172.21.15.70:0/2286306876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34f80953e0 0x7f34f80957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.455+0000 7f34fdf02640 1 --2- 172.21.15.70:0/2286306876 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f34f806d190 0x7f34f8132440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009dc0 tx=0x7f34f002a510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b192bb376bbd40 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.455+0000 7f34effff640 1 -- 172.21.15.70:0/2286306876 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0007740 con 0x7f34f806d190 2024-09-13T15:44:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.455+0000 7f34effff640 1 -- 172.21.15.70:0/2286306876 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f002d050 con 0x7f34f806d190 2024-09-13T15:44:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.455+0000 7f34effff640 1 -- 172.21.15.70:0/2286306876 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0007ac0 con 0x7f34f806d190 2024-09-13T15:44:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.455+0000 7f34ff965640 1 -- 172.21.15.70:0/2286306876 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f34f806d190 msgr2=0x7f34f8132440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.455+0000 7f34ff965640 1 --2- 172.21.15.70:0/2286306876 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f34f806d190 0x7f34f8132440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009dc0 tx=0x7f34f002a510 comp rx=0 tx=0).stop 2024-09-13T15:44:31.459 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.456+0000 7f34ff965640 1 -- 172.21.15.70:0/2286306876 shutdown_connections 2024-09-13T15:44:31.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.456+0000 7f34ff965640 1 --2- 172.21.15.70:0/2286306876 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f34f806d190 0x7f34f8132440 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.456+0000 7f34ff965640 1 --2- 172.21.15.70:0/2286306876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34f80953e0 0x7f34f80957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.456+0000 7f34ff965640 1 -- 172.21.15.70:0/2286306876 >> 172.21.15.70:0/2286306876 conn(0x7f34f8098b10 msgr2=0x7f34f80963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:31.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.456+0000 7f34ff965640 1 -- 172.21.15.70:0/2286306876 shutdown_connections 2024-09-13T15:44:31.460 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.456+0000 7f34ff965640 1 -- 172.21.15.70:0/2286306876 wait complete. 2024-09-13T15:44:31.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.456+0000 7f34ff965640 1 Processor -- start 2024-09-13T15:44:31.462 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34ff965640 1 -- start start 2024-09-13T15:44:31.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34ff965640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f34f806d190 0x7f34f806b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:31.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34ff965640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34f806ddf0 0x7f34f806beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:31.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34ff965640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34f80953e0 0x7f34f806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:31.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34ff965640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f8123d20 con 0x7f34f80953e0 2024-09-13T15:44:31.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34ff965640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f8124930 con 0x7f34f806ddf0 2024-09-13T15:44:31.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34ff965640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34f8124b20 con 0x7f34f806d190 2024-09-13T15:44:31.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34fd701640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f34f806d190 0x7f34f806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:31.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34fcf00640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34f806ddf0 0x7f34f806beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:31.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34fd701640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f34f806d190 0x7f34f806b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:49948/0 (socket says 172.21.15.70:49948) 2024-09-13T15:44:31.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34fd701640 1 -- 172.21.15.70:0/832069661 learned_addr learned my addr 172.21.15.70:0/832069661 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:31.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34fdf02640 1 --2- 172.21.15.70:0/832069661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34f80953e0 0x7f34f806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:31.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34fd701640 1 -- 172.21.15.70:0/832069661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34f806ddf0 msgr2=0x7f34f806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:31.463 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34fd701640 1 --2- 172.21.15.70:0/832069661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34f806ddf0 0x7f34f806beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34fd701640 1 -- 172.21.15.70:0/832069661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34f80953e0 msgr2=0x7f34f806c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34fd701640 1 --2- 172.21.15.70:0/832069661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34f80953e0 0x7f34f806c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34fd701640 1 -- 172.21.15.70:0/832069661 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f4009400 con 0x7f34f806d190 2024-09-13T15:44:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.457+0000 7f34fdf02640 1 --2- 172.21.15.70:0/832069661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34f80953e0 0x7f34f806c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.458+0000 7f34fd701640 1 --2- 172.21.15.70:0/832069661 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f34f806d190 0x7f34f806b970 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f34f400ec10 tx=0x7f34f400c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.458+0000 7f34edffb640 1 -- 172.21.15.70:0/832069661 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4009760 con 0x7f34f806d190 2024-09-13T15:44:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.458+0000 7f34edffb640 1 -- 172.21.15.70:0/832069661 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f400cc30 con 0x7f34f806d190 2024-09-13T15:44:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.458+0000 7f34edffb640 1 -- 172.21.15.70:0/832069661 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4013470 con 0x7f34f806d190 2024-09-13T15:44:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.458+0000 7f34ff965640 1 -- 172.21.15.70:0/832069661 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f0009a70 con 0x7f34f806d190 2024-09-13T15:44:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.458+0000 7f34ff965640 1 -- 172.21.15.70:0/832069661 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f8095f10 con 0x7f34f806d190 2024-09-13T15:44:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.459+0000 7f34edffb640 1 -- 172.21.15.70:0/832069661 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f34f40135d0 con 0x7f34f806d190 2024-09-13T15:44:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.459+0000 7f34ff965640 1 -- 172.21.15.70:0/832069661 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f812b3d0 con 0x7f34f806d190 2024-09-13T15:44:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.459+0000 7f34edffb640 1 -- 172.21.15.70:0/832069661 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f34f405cf50 con 0x7f34f806d190 2024-09-13T15:44:31.465 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.463+0000 7f34edffb640 1 -- 172.21.15.70:0/832069661 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f34f403a990 con 0x7f34f806d190 2024-09-13T15:44:31.599 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.597+0000 7f34ff965640 1 -- 172.21.15.70:0/832069661 --> v1:172.21.15.70:6816/1229510400 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f34f8132980 con 0x7f34dc047560 2024-09-13T15:44:31.600 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.598+0000 7f34edffb640 1 -- 172.21.15.70:0/832069661 <== mgr.24114 v1:172.21.15.70:6816/1229510400 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+34932 (unknown 2965378022 0 3801200093) 0x7f34e8001380 con 0x7f34dc047560 2024-09-13T15:44:31.601 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-13T15:44:31.601 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-13T15:44:31.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.600+0000 7f34ff965640 1 -- 172.21.15.70:0/832069661 >> v1:172.21.15.70:6816/1229510400 conn(0x7f34dc047560 legacy=0x7f34dc049a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:31.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.600+0000 7f34ff965640 1 -- 172.21.15.70:0/832069661 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f34f806d190 msgr2=0x7f34f806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:31.602 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.600+0000 7f34ff965640 1 --2- 172.21.15.70:0/832069661 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f34f806d190 0x7f34f806b970 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f34f400ec10 tx=0x7f34f400c520 comp rx=0 tx=0).stop 2024-09-13T15:44:31.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.600+0000 7f34ff965640 1 -- 172.21.15.70:0/832069661 shutdown_connections 2024-09-13T15:44:31.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.600+0000 7f34ff965640 1 --2- 172.21.15.70:0/832069661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34f80953e0 0x7f34f806c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.600+0000 7f34ff965640 1 --2- 172.21.15.70:0/832069661 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f34f806ddf0 0x7f34f806beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.600+0000 7f34ff965640 1 --2- 172.21.15.70:0/832069661 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f34f806d190 0x7f34f806b970 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.600+0000 7f34ff965640 1 -- 172.21.15.70:0/832069661 >> 172.21.15.70:0/832069661 conn(0x7f34f8098b10 msgr2=0x7f34f8127340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:31.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.600+0000 7f34ff965640 1 -- 172.21.15.70:0/832069661 shutdown_connections 2024-09-13T15:44:31.603 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.600+0000 7f34ff965640 1 -- 172.21.15.70:0/832069661 wait complete. 2024-09-13T15:44:31.613 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-09-13T15:44:31.545531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":101,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":128976,"kb_used_data":2224,"kb_used_omap":17,"kb_used_meta":126702,"kb_avail":377358384,"statfs":{"total":386547056640,"available":386414985216,"internally_reserved":0,"allocated":2277376,"data_stored":1562038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18187,"internal_metadata":129743093},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002523"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T15:44:23.879553+0000","last_change":"2024-09-13T15:44:23.879680+0000","last_active":"2024-09-13T15:44:23.879553+0000","last_peered":"2024-09-13T15:44:23.879553+0000","last_clean":"2024-09-13T15:44:23.879553+0000","last_became_active":"2024-09-13T15:43:56.791454+0000","last_became_peered":"2024-09-13T15:43:56.791454+0000","last_unstale":"2024-09-13T15:44:23.879553+0000","last_undegraded":"2024-09-13T15:44:23.879553+0000","last_fullsized":"2024-09-13T15:44:23.879553+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:44:23.879527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:44:23.879527+0000","last_clean_scrub_stamp":"2024-09-13T15:44:23.879527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:44:04.431608+0000","scrub_duration":0.001538906,"objects_trimmed":0,"snaptrim_duration":0.00035985100000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T15:44:16.905315+0000","last_change":"2024-09-13T15:44:16.905449+0000","last_active":"2024-09-13T15:44:16.905315+0000","last_peered":"2024-09-13T15:44:16.905315+0000","last_clean":"2024-09-13T15:44:16.905315+0000","last_became_active":"2024-09-13T15:43:54.573978+0000","last_became_peered":"2024-09-13T15:43:54.573978+0000","last_unstale":"2024-09-13T15:44:16.905315+0000","last_undegraded":"2024-09-13T15:44:16.905315+0000","last_fullsized":"2024-09-13T15:44:16.905315+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:44:16.905281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:44:16.905281+0000","last_clean_scrub_stamp":"2024-09-13T15:44:16.905281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:01.873456+0000","scrub_duration":0.001353655,"objects_trimmed":0,"snaptrim_duration":0.00054553999999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T15:44:24.890362+0000","last_change":"2024-09-13T15:44:24.890492+0000","last_active":"2024-09-13T15:44:24.890362+0000","last_peered":"2024-09-13T15:44:24.890362+0000","last_clean":"2024-09-13T15:44:24.890362+0000","last_became_active":"2024-09-13T15:43:56.791789+0000","last_became_peered":"2024-09-13T15:43:56.791789+0000","last_unstale":"2024-09-13T15:44:24.890362+0000","last_undegraded":"2024-09-13T15:44:24.890362+0000","last_fullsized":"2024-09-13T15:44:24.890362+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:44:24.890340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:44:24.890340+0000","last_clean_scrub_stamp":"2024-09-13T15:44:24.890340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:11:18.394967+0000","scrub_duration":0.0015259760000000001,"objects_trimmed":0,"snaptrim_duration":0.0011766770000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T15:44:18.874560+0000","last_change":"2024-09-13T15:44:18.874667+0000","last_active":"2024-09-13T15:44:18.874560+0000","last_peered":"2024-09-13T15:44:18.874560+0000","last_clean":"2024-09-13T15:44:18.874560+0000","last_became_active":"2024-09-13T15:43:54.573540+0000","last_became_peered":"2024-09-13T15:43:54.573540+0000","last_unstale":"2024-09-13T15:44:18.874560+0000","last_undegraded":"2024-09-13T15:44:18.874560+0000","last_fullsized":"2024-09-13T15:44:18.874560+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:44:18.874536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:44:18.874536+0000","last_clean_scrub_stamp":"2024-09-13T15:44:18.874536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:30:32.944812+0000","scrub_duration":0.001439383,"objects_trimmed":0,"snaptrim_duration":0.00024452499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"32'3","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T15:44:16.811749+0000","last_change":"2024-09-13T15:44:16.811877+0000","last_active":"2024-09-13T15:44:16.811749+0000","last_peered":"2024-09-13T15:44:16.811749+0000","last_clean":"2024-09-13T15:44:16.811749+0000","last_became_active":"2024-09-13T15:43:56.584517+0000","last_became_peered":"2024-09-13T15:43:56.584517+0000","last_unstale":"2024-09-13T15:44:16.811749+0000","last_undegraded":"2024-09-13T15:44:16.811749+0000","last_fullsized":"2024-09-13T15:44:16.811749+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3","last_scrub_stamp":"2024-09-13T15:44:16.811726+0000","last_deep_scrub":"32'3","last_deep_scrub_stamp":"2024-09-13T15:44:16.811726+0000","last_clean_scrub_stamp":"2024-09-13T15:44:16.811726+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:47:23.646604+0000","scrub_duration":0.003013253,"objects_trimmed":0,"snaptrim_duration":0.00094542300000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T15:44:14.630673+0000","last_change":"2024-09-13T15:44:14.630771+0000","last_active":"2024-09-13T15:44:14.630673+0000","last_peered":"2024-09-13T15:44:14.630673+0000","last_clean":"2024-09-13T15:44:14.630673+0000","last_became_active":"2024-09-13T15:43:56.582550+0000","last_became_peered":"2024-09-13T15:43:56.582550+0000","last_unstale":"2024-09-13T15:44:14.630673+0000","last_undegraded":"2024-09-13T15:44:14.630673+0000","last_fullsized":"2024-09-13T15:44:14.630673+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:44:14.630652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:44:14.630652+0000","last_clean_scrub_stamp":"2024-09-13T15:44:14.630652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:31.222349+0000","scrub_duration":0.0013855709999999999,"objects_trimmed":0,"snaptrim_duration":0.00089033199999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T15:44:26.895699+0000","last_change":"2024-09-13T15:44:26.895838+0000","last_active":"2024-09-13T15:44:26.895699+0000","last_peered":"2024-09-13T15:44:26.895699+0000","last_clean":"2024-09-13T15:44:26.895699+0000","last_became_active":"2024-09-13T15:43:56.584633+0000","last_became_peered":"2024-09-13T15:43:56.584633+0000","last_unstale":"2024-09-13T15:44:26.895699+0000","last_undegraded":"2024-09-13T15:44:26.895699+0000","last_fullsized":"2024-09-13T15:44:26.895699+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T15:44:26.895678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T15:44:26.895678+0000","last_clean_scrub_stamp":"2024-09-13T15:44:26.895678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:32.969480+0000","scrub_duration":0.0016311819999999999,"objects_trimmed":0,"snaptrim_duration":0.00034895899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"32'2","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T15:44:25.885883+0000","last_change":"2024-09-13T15:44:25.885986+0000","last_active":"2024-09-13T15:44:25.885883+0000","last_peered":"2024-09-13T15:44:25.885883+0000","last_clean":"2024-09-13T15:44:25.885883+0000","last_became_active":"2024-09-13T15:43:56.582549+0000","last_became_peered":"2024-09-13T15:43:56.582549+0000","last_unstale":"2024-09-13T15:44:25.885883+0000","last_undegraded":"2024-09-13T15:44:25.885883+0000","last_fullsized":"2024-09-13T15:44:25.885883+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2","last_scrub_stamp":"2024-09-13T15:44:25.885859+0000","last_deep_scrub":"32'2","last_deep_scrub_stamp":"2024-09-13T15:44:25.885859+0000","last_clean_scrub_stamp":"2024-09-13T15:44:25.885859+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:12:40.881791+0000","scrub_duration":0.003055239,"objects_trimmed":0,"snaptrim_duration":0.0010276560000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"32'65","reported_seq":154,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-13T15:44:14.875784+0000","last_change":"2024-09-13T15:44:14.875919+0000","last_active":"2024-09-13T15:44:14.875784+0000","last_peered":"2024-09-13T15:44:14.875784+0000","last_clean":"2024-09-13T15:44:14.875784+0000","last_became_active":"2024-09-13T15:43:54.574044+0000","last_became_peered":"2024-09-13T15:43:54.574044+0000","last_unstale":"2024-09-13T15:44:14.875784+0000","last_undegraded":"2024-09-13T15:44:14.875784+0000","last_fullsized":"2024-09-13T15:44:14.875784+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'65","last_scrub_stamp":"2024-09-13T15:44:14.875763+0000","last_deep_scrub":"32'65","last_deep_scrub_stamp":"2024-09-13T15:44:14.875763+0000","last_clean_scrub_stamp":"2024-09-13T15:44:14.875763+0000","objects_scrubbed":2,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:27.596937+0000","scrub_duration":0.0042116949999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":152,"num_read_kb":252,"num_write":99,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":28,"seq":120259084297,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":268,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":274432,"data_stored":95335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]}]},{"osd":2,"up_from":27,"seq":115964117001,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":264,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":270336,"data_stored":95316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]}]},{"osd":1,"up_from":26,"seq":111669149705,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":848,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":868352,"data_stored":685703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4546,"internal_metadata":32435774},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]}]},{"osd":0,"up_from":25,"seq":107374182409,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":844,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":864256,"data_stored":685684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4547,"internal_metadata":32435773},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T15:44:31.614 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T15:44:31.712 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.710+0000 7f650f3ec640 1 Processor -- start 2024-09-13T15:44:31.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.710+0000 7f650f3ec640 1 -- start start 2024-09-13T15:44:31.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.710+0000 7f650f3ec640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f650806ca80 0x7f650806ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:31.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.710+0000 7f650f3ec640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65080957a0 0x7f6508132610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:31.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.710+0000 7f650f3ec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f650806e310 con 0x7f650806d3a0 2024-09-13T15:44:31.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.710+0000 7f650f3ec640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650806dfc0 con 0x7f65080957a0 2024-09-13T15:44:31.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.710+0000 7f650f3ec640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6508097d40 con 0x7f650806ca80 2024-09-13T15:44:31.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.710+0000 7f650d188640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f650806ca80 0x7f650806ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:31.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.710+0000 7f650d989640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65080957a0 0x7f6508132610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:31.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.711+0000 7f650d188640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f650806ca80 0x7f650806ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:49952/0 (socket says 172.21.15.70:49952) 2024-09-13T15:44:31.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.710+0000 7f650c987640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f650806d3a0 0x7f6508095090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37806/0 (socket says 172.21.15.70:37806) 2024-09-13T15:44:31.713 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.711+0000 7f650d188640 1 -- 172.21.15.70:0/1546169440 learned_addr learned my addr 172.21.15.70:0/1546169440 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:31.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.711+0000 7f650d188640 1 -- 172.21.15.70:0/1546169440 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65080957a0 msgr2=0x7f6508132610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T15:44:31.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.711+0000 7f650d188640 1 --2- 172.21.15.70:0/1546169440 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65080957a0 0x7f6508132610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.718 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.711+0000 7f650d188640 1 -- 172.21.15.70:0/1546169440 >> v1:172.21.15.70:6789/0 conn(0x7f650806d3a0 legacy=0x7f6508095090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:31.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.711+0000 7f650d188640 1 -- 172.21.15.70:0/1546169440 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650806f550 con 0x7f650806ca80 2024-09-13T15:44:31.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.711+0000 7f64fffff640 1 -- 172.21.15.70:0/1546169440 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101266614 0 0) 0x7f650806e310 con 0x7f650806d3a0 2024-09-13T15:44:31.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.711+0000 7f650d989640 1 --2- 172.21.15.70:0/1546169440 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65080957a0 0x7f6508132610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T15:44:31.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.711+0000 7f650d188640 1 --2- 172.21.15.70:0/1546169440 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f650806ca80 0x7f650806ce60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64f80098d0 tx=0x7f64f802a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=300d9fcddcb75023 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:31.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.712+0000 7f64fffff640 1 -- 172.21.15.70:0/1546169440 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f802ae50 con 0x7f650806ca80 2024-09-13T15:44:31.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.712+0000 7f64fffff640 1 -- 172.21.15.70:0/1546169440 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f8032440 con 0x7f650806ca80 2024-09-13T15:44:31.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.712+0000 7f64fffff640 1 -- 172.21.15.70:0/1546169440 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f80327d0 con 0x7f650806ca80 2024-09-13T15:44:31.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.712+0000 7f650f3ec640 1 -- 172.21.15.70:0/1546169440 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f650806ca80 msgr2=0x7f650806ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:31.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.712+0000 7f650f3ec640 1 --2- 172.21.15.70:0/1546169440 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f650806ca80 0x7f650806ce60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64f80098d0 tx=0x7f64f802a140 comp rx=0 tx=0).stop 2024-09-13T15:44:31.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.712+0000 7f650f3ec640 1 -- 172.21.15.70:0/1546169440 shutdown_connections 2024-09-13T15:44:31.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.712+0000 7f650f3ec640 1 --2- 172.21.15.70:0/1546169440 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f65080957a0 0x7f6508132610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.712+0000 7f650f3ec640 1 --2- 172.21.15.70:0/1546169440 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f650806ca80 0x7f650806ce60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.719 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.712+0000 7f650f3ec640 1 -- 172.21.15.70:0/1546169440 >> 172.21.15.70:0/1546169440 conn(0x7f6508098b40 msgr2=0x7f65080963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:31.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.712+0000 7f650f3ec640 1 -- 172.21.15.70:0/1546169440 shutdown_connections 2024-09-13T15:44:31.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.712+0000 7f650f3ec640 1 -- 172.21.15.70:0/1546169440 wait complete. 2024-09-13T15:44:31.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.713+0000 7f650f3ec640 1 Processor -- start 2024-09-13T15:44:31.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.713+0000 7f650f3ec640 1 -- start start 2024-09-13T15:44:31.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.713+0000 7f650f3ec640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f650806ca80 0x7f6508132f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:31.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.713+0000 7f650f3ec640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f650806d3a0 0x7f6508124680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:31.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.713+0000 7f650f3ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65080957a0 0x7f6508124bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T15:44:31.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.713+0000 7f650f3ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65081334b0 con 0x7f65080957a0 2024-09-13T15:44:31.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.713+0000 7f650f3ec640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650812c820 con 0x7f650806ca80 2024-09-13T15:44:31.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.713+0000 7f650f3ec640 1 -- --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6508126090 con 0x7f650806d3a0 2024-09-13T15:44:31.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f650d188640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f650806ca80 0x7f6508132f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:31.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f650c987640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f650806d3a0 0x7f6508124680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:31.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f650d989640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65080957a0 0x7f6508124bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T15:44:31.720 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f650c987640 1 --2- >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f650806d3a0 0x7f6508124680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3302/0 says I am v2:172.21.15.70:49954/0 (socket says 172.21.15.70:49954) 2024-09-13T15:44:31.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f650c987640 1 -- 172.21.15.70:0/694197451 learned_addr learned my addr 172.21.15.70:0/694197451 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-13T15:44:31.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f650c987640 1 -- 172.21.15.70:0/694197451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f650806ca80 msgr2=0x7f6508132f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:31.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f650c987640 1 --2- 172.21.15.70:0/694197451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f650806ca80 0x7f6508132f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f650c987640 1 -- 172.21.15.70:0/694197451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65080957a0 msgr2=0x7f6508124bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:31.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f650c987640 1 --2- 172.21.15.70:0/694197451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65080957a0 0x7f6508124bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f650c987640 1 -- 172.21.15.70:0/694197451 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6500009a70 con 0x7f650806d3a0 2024-09-13T15:44:31.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f650c987640 1 --2- 172.21.15.70:0/694197451 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f650806d3a0 0x7f6508124680 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f650400c4a0 tx=0x7f650400c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T15:44:31.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f64fdffb640 1 -- 172.21.15.70:0/694197451 <== mon.2 v2:172.21.15.70:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65040074e0 con 0x7f650806d3a0 2024-09-13T15:44:31.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f64fdffb640 1 -- 172.21.15.70:0/694197451 <== mon.2 v2:172.21.15.70:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6504007640 con 0x7f650806d3a0 2024-09-13T15:44:31.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f650f3ec640 1 -- 172.21.15.70:0/694197451 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f80094d0 con 0x7f650806d3a0 2024-09-13T15:44:31.721 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f64fdffb640 1 -- 172.21.15.70:0/694197451 <== mon.2 v2:172.21.15.70:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504004a20 con 0x7f650806d3a0 2024-09-13T15:44:31.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.714+0000 7f650f3ec640 1 -- 172.21.15.70:0/694197451 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650812bde0 con 0x7f650806d3a0 2024-09-13T15:44:31.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.715+0000 7f650f3ec640 1 -- 172.21.15.70:0/694197451 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6508095e40 con 0x7f650806d3a0 2024-09-13T15:44:31.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.716+0000 7f64fdffb640 1 -- 172.21.15.70:0/694197451 <== mon.2 v2:172.21.15.70:3302/0 4 ==== mgrmap(e 10) v1 ==== 55212+0+0 (secure 0 0 0) 0x7f6504004b80 con 0x7f650806d3a0 2024-09-13T15:44:31.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.716+0000 7f64fdffb640 1 -- 172.21.15.70:0/694197451 <== mon.2 v2:172.21.15.70:3302/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3027+0+0 (secure 0 0 0) 0x7f650405c660 con 0x7f650806d3a0 2024-09-13T15:44:31.722 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.719+0000 7f64fdffb640 1 -- 172.21.15.70:0/694197451 <== mon.2 v2:172.21.15.70:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+146310 (secure 0 0 0) 0x7f650403a2d0 con 0x7f650806d3a0 2024-09-13T15:44:31.856 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.854+0000 7f650f3ec640 1 -- 172.21.15.70:0/694197451 --> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f650806ea40 con 0x7f650806d3a0 2024-09-13T15:44:31.859 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.858+0000 7f64fdffb640 1 -- 172.21.15.70:0/694197451 <== mon.2 v2:172.21.15.70:3302/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f650403a2d0 con 0x7f650806d3a0 2024-09-13T15:44:31.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.859+0000 7f64fdffb640 1 -- 172.21.15.70:0/694197451 <== mon.2 v2:172.21.15.70:3302/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) v1 ==== 123+0+0 (secure 0 0 0) 0x7f650403d8f0 con 0x7f650806d3a0 2024-09-13T15:44:31.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.861+0000 7f650f3ec640 1 -- 172.21.15.70:0/694197451 >> v1:172.21.15.70:6816/1229510400 conn(0x7f64ec047590 legacy=0x7f64ec049a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:31.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.861+0000 7f650f3ec640 1 -- 172.21.15.70:0/694197451 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f650806d3a0 msgr2=0x7f6508124680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T15:44:31.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.861+0000 7f650f3ec640 1 --2- 172.21.15.70:0/694197451 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f650806d3a0 0x7f6508124680 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f650400c4a0 tx=0x7f650400c8a0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.861+0000 7f650f3ec640 1 -- 172.21.15.70:0/694197451 shutdown_connections 2024-09-13T15:44:31.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.861+0000 7f650f3ec640 1 --2- 172.21.15.70:0/694197451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65080957a0 0x7f6508124bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.861+0000 7f650f3ec640 1 --2- 172.21.15.70:0/694197451 >> [v2:172.21.15.70:3302/0,v1:172.21.15.70:6791/0] conn(0x7f650806d3a0 0x7f6508124680 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.861+0000 7f650f3ec640 1 --2- 172.21.15.70:0/694197451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f650806ca80 0x7f6508132f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T15:44:31.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.861+0000 7f650f3ec640 1 -- 172.21.15.70:0/694197451 >> 172.21.15.70:0/694197451 conn(0x7f6508098b40 msgr2=0x7f65080971c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T15:44:31.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.861+0000 7f650f3ec640 1 -- 172.21.15.70:0/694197451 shutdown_connections 2024-09-13T15:44:31.864 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-13T15:44:31.861+0000 7f650f3ec640 1 -- 172.21.15.70:0/694197451 wait complete. 2024-09-13T15:44:31.874 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T15:44:31.874 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T15:44:31.874 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T15:44:31.875 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:44:31.908 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T15:44:31.908 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T15:44:31.909 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:44:31.983 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T15:44:31.983 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T15:44:31.983 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:44:32.052 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T15:44:32.052 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T15:44:32.052 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:44:32.125 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T15:44:32.125 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T15:44:32.125 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T15:44:32.125 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:44:32.185 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T15:44:32.186 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T15:44:32.186 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:44:32.252 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T15:44:32.252 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T15:44:32.253 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T15:44:32.253 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:44:32.304 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T15:44:32.304 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T15:44:32.304 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:44:32.355 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T15:44:32.355 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T15:44:32.355 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T15:44:32.406 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T15:44:32.407 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T15:44:32.407 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(FS_' | egrep -v '\(MDS_' | egrep -v '\(OSD_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(CACHE_POOL_' | egrep -v '\(POOL_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v 'Monitor daemon marked osd' | egrep -v 'Behind on trimming' | egrep -v 'Manager daemon' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'scrub mismatch' | egrep -v ScrubResult | egrep -v 'wrongly marked' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_MSGR2_NOT_ENABLED\)' | egrep -v 'slow request' | head -n 1 2024-09-13T15:44:32.487 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T15:44:32.487 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T15:44:32.667 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T15:44:32.668 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T15:44:32.748 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T15:44:32.749 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T15:44:32.805 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T15:44:32.806 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T15:44:32.865 INFO:tasks.ceph:Archiving mon data... 2024-09-13T15:44:32.866 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903753/data/mon.a.tgz 2024-09-13T15:44:32.866 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-13T15:44:32.881 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.QaYQJrupQK 2024-09-13T15:44:32.882 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.QaYQJrupQK 2024-09-13T15:44:33.001 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.QaYQJrupQK 2024-09-13T15:44:33.083 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.QaYQJrupQK is 218KB 2024-09-13T15:44:33.177 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.QaYQJrupQK 2024-09-13T15:44:33.192 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903753/data/mon.b.tgz 2024-09-13T15:44:33.192 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-13T15:44:33.246 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.6SYKasFWdp 2024-09-13T15:44:33.246 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.6SYKasFWdp 2024-09-13T15:44:33.358 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.6SYKasFWdp 2024-09-13T15:44:33.441 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.6SYKasFWdp is 257KB 2024-09-13T15:44:33.535 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.6SYKasFWdp 2024-09-13T15:44:33.549 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903753/data/mon.c.tgz 2024-09-13T15:44:33.550 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-13T15:44:33.604 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.NVkBn42q4Q 2024-09-13T15:44:33.604 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.NVkBn42q4Q 2024-09-13T15:44:33.717 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.NVkBn42q4Q 2024-09-13T15:44:33.799 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.NVkBn42q4Q is 257KB 2024-09-13T15:44:33.893 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.NVkBn42q4Q 2024-09-13T15:44:33.908 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T15:44:33.908 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T15:44:33.972 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T15:44:33.972 DEBUG:teuthology.orchestra.run.smithi070:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T15:44:34.037 INFO:teuthology.orchestra.run.smithi070.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T15:44:34.042 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T15:44:34.044 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903753/remote/smithi070/crash 2024-09-13T15:44:34.045 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T15:44:34.106 INFO:tasks.ceph:Compressing logs... 2024-09-13T15:44:34.106 DEBUG:teuthology.orchestra.run.smithi070:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T15:44:34.711 INFO:tasks.ceph:Archiving logs... 2024-09-13T15:44:34.711 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903753/remote/smithi070/log 2024-09-13T15:44:34.712 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T15:44:34.895 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T15:44:34.913 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T15:44:34.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T15:44:34.972 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T15:44:34.984 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T15:44:34.984 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T15:44:34.999 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-09-13T15:44:35.003 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T15:44:35.003 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-09-13T15:44:35.003 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv01.front.sepia.ceph.com 3 6 177 59 +286us[ +274us] +/- 45ms 2024-09-13T15:44:35.003 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv02.front.sepia.ceph.com 3 6 177 59 -758us[ -770us] +/- 61ms 2024-09-13T15:44:35.003 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 6 177 59 +421us[ +421us] +/- 72ms 2024-09-13T15:44:35.003 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T15:44:35.004 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T15:44:35.015 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T15:44:35.016 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T15:44:35.026 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T15:44:35.108 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T15:44:35.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T15:44:35.136 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T15:44:35.148 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T15:44:35.158 INFO:teuthology.task.internal:Duration was 485.774421 seconds 2024-09-13T15:44:35.159 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T15:44:35.169 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T15:44:35.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T15:44:35.214 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T15:44:35.459 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T15:44:35.460 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2024-09-13T15:44:35.460 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T15:44:35.521 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T15:44:35.521 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T15:44:35.587 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T15:44:35.587 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T15:44:35.728 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T15:44:35.739 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T15:44:35.740 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T15:44:35.765 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T15:44:35.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T15:44:35.828 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2024-09-13T15:44:35.848 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T15:44:35.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T15:44:35.903 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T15:44:35.914 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T15:44:35.915 DEBUG:teuthology.misc:Transferring archived files from smithi070:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:36:41-fs-wip-rishabh-testing-20240910.095512-reef-distro-default-smithi/7903753/remote/smithi070 2024-09-13T15:44:35.915 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T15:44:35.977 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T15:44:35.978 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T15:44:36.027 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T15:44:36.041 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T15:44:36.041 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T15:44:36.053 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T15:44:36.053 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T15:44:36.082 INFO:teuthology.orchestra.run.smithi070.stdout: 266088 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 15:44 /home/ubuntu/cephtest 2024-09-13T15:44:36.083 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T15:44:36.095 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T15:44:36.120 INFO:teuthology.run:Summary data: description: fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/quincy 1-upgrade}} duration: 485.7744209766388 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-13T15:44:36.120 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T15:44:36.221 INFO:teuthology.run:pass